After my commentary below are annotated links to more than thirty of Avi Bryant’s public presentations, blog posts and interviews, along with associated videos, podcasts, transcripts, slides and so on.
You can skip my introductory commentary and see (except for recent and prospective talks) the comprehensive list of Avi Bryant’s talks directly.
Some slightly recent talks
• Adding Tree and Tree: Distributed Decision Tree Learning November 15, 2014 – 9:00 AM
PNWScala 2014 – Pacific Northwest SCALA Conference (November 14-15, 2014)
Portland Oregon USA
Video: “Brushfire: Distributed, Generic, Tree Learning in Scala (using Hadoop)”
“Brushfire is a framework for distributed, supervised learning of ensembles of decision trees. It is designed to be extremely scalable (both in number of observations and number of features) and extremely customizable: it makes heavy use of Scala generics and typeclasses to allow easy parameterization of each stage of the tree building algorithm. This talk will walk through the overall architecture and algorithm, then show some simple concrete examples of using Brushfire with Scalding to build trees on Hadoop.”
• Mayday.us (Avi Bryant’s first Strip.com blog post, November 4, 2014.)
• Getting Started with Scalding, Twitter’s High-level Scala API for Hadoop MapReduce
1:30pm Monday, July 21, 2014,
OSCON – Open Source Convention (July 20-24, 2014)
Portland, Oregon USA
From the tutorial description:
Start on low heat with a base of Hadoop; map, then reduce. Flavor, to taste, with Scala’s concise, functional syntax and collections library. Simmer with some Pig bones: a tuple model and high-level join and aggregation operators. Mix in Cascading to hold everything together and boil until it’s very, very hot, and you get Scalding, an API for MapReduce out of Twitter.
Scalding is an open source Scala framework for concisely describing Hadoop MapReduce jobs. I started the project at Twitter as a way for ad server engineers to run simple queries on the ad logs, without needing to learn a specialized language like Pig, or dive too deeply into the guts of Hadoop. Since then, it’s been adopted by teams at Etsy, LinkedIn, EBay, SoundCloud, LivePerson, Stripe, and others, and been extended with convenient APIs for everything from large-scale sparse matrix multiplication to locality-sensitive hashing.
This tutorial will walk you through getting started with Scalding, from writing the simplest word-count job up to using probabilistic data structures for distributed machine learning. No specific background in Scala, Hadoop, distributed computing or machine learning is required, though an interest in any or all of these might help.
No specific knowledge needed. Some familiarity with either Scala or Hadoop would be helpful but is not at all required. Bring a laptop with a working JDK installation.
• CUSEC 2014 January 16-18, 2014
Canadian University Software Engineering Conference
Montreal, Quebec, Canada
A talk without title, on Friday, January 17, according to the CUSEC 2014 schedule
Video, on software engineering, counting, probabilities and hashing: https://vimeo.com/115733572 (45 minutes)
• Add ALL the things: abstract algebra meets analytics Avi Bryant
Strange Loop 2013 (September 18-20, 2013) Saint Louis, Missouri USA
Video: Add ALL the Things: Abstract Algebra Meets Analytics (38 minutes)
InfoQ’s videos for the entire Strange Loop Conference.
Slides for “Add ALL the Things” are available via Github, as are most of the other StrangeLoop 2013 session slides.
Also Slides & videos for Emerging Languages Camp at Strange Loop 2013 are on InfoQ as well. — —
• On October 25, 2012, at that time the lead for six-months of an expanded Etsy data analytics group, Bryant presented in New York City at the O’Reilly Strata + Hadoop World conference, with a talk entitled Scala + Cascading = Scalding. Leading up to that presentation, he posted to the O’Reilly Strata site an article entitled MATLAB, R, and Julia: Languages for data analysis: Inside core features of specialized data analysis languages on October 15, 2012. The Strata + Hadoop World conference was part of New York City Data Week, running from October 22 – 26, 2012.
• In early February 2013, he gave a talk entitled scala.collection.approximate, during Northeast Scala Symposium at Drexel University in Philadelphia, which was held February 8 & 9, 2013. (See also @nescalas). The video of Avi’s talk was released on March 21st. All of the conference’s presentations were recorded, and the other presentation videos are available. See Marakana Inc.’s 2013 Northeast Scala Symposium video release page, and the list of conference videos released to YouTube. (Twitter announced on August 13, 2013 that it had bought out Marakana to create its own training and video services department.) The prior year’s presentation videos for 2012 Northeast Scala Symposium (in Boston) were put online, as were the 2011 NESCALA presentations.
Avi’s synopsis of his Northeast Scala Symposium talk:
“How do you count the number of distinct values when there’s too many to fit in a Set? How do you find the most frequent items when you don’t have space for a Map[T,Int]? When dealing with very large or streaming datasets, it’s common to use probabilistic algorithms like bloom filters, hyperloglog counters, or count min sketches. In Scala, it’s nice to model these as approximate versions of normal collection classes like Set and Map. I’ll discuss some work we’re doing on this in the context of the Algebird project started at Twitter.”
• Add ALL the things: Abstract Algebra and Distributed Data Processing (38 minutes)
A presentation made in New York to Kickstarter developers, prior to Avi’s departure from New York-based Etsy.
(Posted Apr 22, 2013)
“Anyone who builds analytics systems ends up understanding abelian groups, whether they know it or not. This is a talk I gave at Kickstarter about how the laws of group theory provide a useful codification of the practical lessons of building efficient distributed and realtime aggregation systems.”
Projects and code that make use of the ideas described in the talk:
– Algebird – Abstract Algebra for Scala
– Simmer: a streaming aggregation tool inspired in part by Hadoop streaming’s Aggregate package, but uses the probabalistic aggregation algorithms from Twitter’s Algebird.
The link to the list of the rest of Avi Bryant’s talks, further below.
Below, I survey Avi Bryant’s description (2009 through 2011) of the expanding client-side opportunities available to designers of web application frameworks, along with a (rather too long) list of posts by others surveying aspects of the topic. The subject of frameworks is only one of many threads and challenges issued in Avi’s public talks, which run from 2001 through 2013.
Bryant’s previous blog presence went dark upon his departure from Twitter on December 2, 2011. Bryant announced on a new blog, on April 17, 2012, his plan to move from San Francisco and return to Galiano Island, near Vancouver Canada to work remotely for New York-based Etsy, starting in June 2012. He hired and launched and led for Etsy a geographically distributed data analytics team to collaborate with.
On March 11, 2013, he announced that he had joined up with friends that had started San Francisco-based Stripe, and that he would continue to work remotely from Galiano Island.
Context and Background
In 2005, Avi Bryant co-founded Smallthought Systems, Inc., along with partner and co-CEO Andrew Catton. Smallthought Systems was based in Vancouver, British Columbia, Canada. Bryant and Catton were subsequently joined in collaboration by Luke Andrews and Ben Matasar, and together they produced several public Smalltalk-based web applications in addition to private client projects.
Smallthought System’s first public web application was the online database DabbleDB. It was first exposed for public beta on October 24, 2005, and received active feature improvement and bug elimination effort during the next three years or so. The DabbleDB demo video narrated by Avi Bryant and can still be found online (archived by other admirers): DabbleDB Demo on Youtube.
Seaside Smalltalk web application framework
DabbleDB was implemented using Seaside, a template-free web application framework written in Smalltalk. Avi Bryant developed Seaside with Julian Fitzell and others and Seaside was open-sourced in 2002. Seaside had been inspired by NeXT/Apple’s WebObjects approach to client browser and server interaction in its original Objective-C and subsequent Java versions. WebObjects focuses on the state and behavior of the application as distinct from the mechanics of URLs and HTML and HTTP requests. Avi prototyped the architectural ideas in Ruby (well before the existence of Ruby on Rails), via the IOWA web application framework, and abandoned IOWA in 2001 when he elected to implement the approach in Smalltalk, as “Seaside”. Bryant has described his language progression in the years before this era as C –> Objective-C –> Ruby –> Smalltalk.
Monticello Smalltalk distributed version control system
Bryant developed the Monticello Smalltalk DVCS in collaboration with Colin Putney and other programmers in the Smalltalk community in 2003.
Smallthought’s financing and later web application projects
Smallthought Systems received a venture capital investment by Ventures West, a Canadian VC firm, in June 2006, a transaction facilitated by Paul Kedrosky. The actual amount of the investment, the portion of the company sold, and the post-transaction valuation of the company was not disclosed by the participants (though one apparently uninformed journalist speculated the investment approached two million dollars).
Smallthought Systems’s second public web application, a Facebook app, was a dud and withdrawn a few months after its introduction. Apparently people did not want to create a to-do list on Facebook. The Dabble Do announcement: Introducing Dabble Do (August 6, 2007)
In 2008 Smallthought released Magic/Replace, an online table-oriented data editing tool. Video demo still online at: http://vimeo.com/2356472
A web traffic analyzer and trend-line display application and interpreter of Google Analytics data called Trendly was released in 2009 (while in beta, it was called “DSHBRD”). The Trendly web application received the attention of the Google Analytics Blog in a post describing Trendly’s use of the Google API, and Trendly’s the then-innovative perspective on trend analysis, in the post An API Integration To Measure Significant Change (August 28, 2009). It is fair to say that the statistical and programming thinking demonstrated in implementing this application led to Twitter’s purchase of Smallthought Systems for its analytical talent.
Smallthought Systems purchased by Twitter
On June 10, 2010, Twitter announced that it had completed its purchase of Smallthought Systems, Inc., a hiring purchase of talent rather than an acquisition of software. All four Smallthought Systems developers joined Twitter as employees to work on the Twitter data analytics team, and moved to San Francisco where for the first time they all resided the same city. Over the course of 2011, the online applications DabbleDB and Trendly were closed down. (Tim Bray reported on August 24, 2011 that he was able to sell his investment in Smallthought Systems at a profit. Bray’s shares had been converted to shares in Twitter and he apparently made the investment on similar terms as Kedroskys Ventures West participation. It hardly needs saying that that implies the founders did well for their share of the company.)
Scalding MapReduce framework
While at Twitter, Bryant led Twitter’s adoption of Scala for Hadoop jobs, and collaborated in developing the in-house Scalding MapReduce API framework (using Scala and Cascading) that Twitter subsequently open-sourced in March 2012. Its Github page describes Scalding as “a Scala library that makes it easy to write MapReduce jobs in Hadoop. Instead of forcing you to write raw map and reduce functions, Scalding allows you to write code that looks like natural Scala.” Scalding continues to be under active development. See a Twitter Engineering blog post in September 2012; Bryant continues to contribute from his position at Etsy. His founding collaborators at Twitter included Oscar Boykin (@posco) and Argyris Zymnis (@argyris).
See the link further below for the video of Bryant’s talk “Smalltalk & Big Data”, for the ideas for the design of an easier-to-use front-end for Hadoop MapReduce tasks using Smalltalk—that became the conceptual foundation and framework for Scalding, using the Twitter in-house language Scala instead of Smalltalk.
Avi Bryant gave a presentation at the Strata Conference + Hadoop World, New York City, October 25, 2012, entitled Scala + Cascading = Scalding. He also gave a talk entitled scala.collection.approximate, during Northeast Scala Symposium at Drexel University in Philadelphia. The conference was February 8-9, 2013.
Avi Bryant departs from Twitter
Avi Bryant announced, via Twitter, his departure as an employee on December 2, 2011. He indicated he would be working on new startups of a rather unspecified nature. Bryant did subsequently acknowlege nearly a year after leaving Twitter that the acquisition/hire transition from the tiny Smallthought Systems to the rapidly growing three-to-four orders-of-magnitude larger Twitter had a number of hard lessons and adjustments for a developer accustomed to nimble development and rapid deployment of new applications to the public.
Server-centric web application frameworks are obsolete
Migrating away from Seaside
At five different conferences, in 2009 and 2010, Bryant described the process of gradually abandoning use of his own path-breaking Seaside Smalltalk web framework during several development and architecture iterations of Smallthought Systems’s Trendly, prior to Trendly’s initial release. The talks were entitled: “1,001 Iterations: Product Design, Illustrated”, “Failure: An Illustrated Guide”, “Django is obsolete (but so is everything else)”, “Rails is obsolete (but so’s everything else)” and “Smalltalk and Big Data”.
• Amber Smalltalk extends Clamato, with a jQuery binding, led by Nicolas Petton. Until the 0.9 release (September 14, 2011), Amber was called JTalk. (In September 2011, Avi Bryant said to Nicolas Petton that the progress to-date on JTalk had eclipsed the initial Clamato effort, which had been fallow for two years at that point.) The Amber project has an active mailing list. Nicolas Petton’s introductory presentation about JTalk / Amber Smalltalk at ESUG 2011 is available as a video, along with the slides, which were written in Amber Smalltalk. Petton also outlines the differences between Amber and other Smalltalk dialects in “Porting code from other Smalltalk dialects“.
• Yet another in-browser Smalltalk dialect project is SilverSmalltalk which has an ExtJS binding, and is a project by Peter Fisk (SilverSmalltalk repository).
Meteor web app framework; Luna web app framework (unreleased: used internally by Asana); Derby.js; Racer.js; Cappuccino; SproutCore; Ember.JS (a renamed fork of SproutCore) [not to be confused with Ender.js]; Node.JS; NowJS; SocketStream; Socket.IO; Express; Backbone.js; Common.js; Knockout.js; JSDOM. And Opa is an example of an open, tierless framework, DSL and web stack, with a compiler written in OCaml; naturally there are a few other tierless framework projects.
You can skip directly to the list of Avi Bryant’s talks.
• Adam Bosworth Interview with Adam Bosworth ACM Queue (March 1, 2003)
• John Gruber The Location Field Is the New Command Line Daring Fireball (June 22, 2004)
• Koranteng Ofosu-Amaah On rich web applications, AlphaBlox and Oddpost (July 16, 2004)
• Adam Bosworth Ajax reconsidered (June 1, 2005)
• Ian Bicking Constraints and Continuations (March 22, 2006)
• Francisco Tolmasky The Road to Cappuccino</a (presentation slides & audio) AJAX Experience Conference, Boston (September 29 – October 1, 2008)
• Patrick Meuller Brainwashed (November 12, 2008)
• Zachary Voase Why JSON will save bioinformatics (well, sort of…) (May 18, 2009)
• Bret Taylor The technology behind Tornado, FriendFeed’s web server (September 10, 2009)
• Chris Williams Going evented with Node.js (November 16, 2009)
• Dan Ingalls Node.JS What’s Up with Dan Ingalls (April 26, 2010)
• Ask HN What are current trends in web developing? (June 2010)
• Trip Hawkins Games need some SAAS (June 20, 2010)
• Charles Jolley The Next Revolution SproutCore Blog (July 1, 2010)
• Yahuda Katz Here’s to the Next 3 Years Katz Got Your Tongue (September 2010)
• Julian Fitzell Seaside – Why should you care? Dynamic Stockholm 2010</a (October 19, 2010)
• John Arley Burns Why Mobile Apps Are Obsolete Rant 3000 (November 18, 2010)
• Allen Wirfs-Brock The Browser is a Transitional Technology (January 31, 2011)
• Gilad Bracha The Truthiness Is Out There Room 101 (March 20, 2011)
•John Hann The Future is Modules not Frameworks (slides), JSConf 2011, Portland Oregon (May 3, 2011)
• Antranig Basman The Future is Frameworks, not Modules (powerpoint slides),JSConf 2011, Portland Oregon (May 4, 2011)
• Brendan Eich My JSConf.US Presentation Slides: CoffeScript, JS.next, the JS Language Lab, JSConf 2011, Portland Oregon (May 3, 2011)
• Allen Wirfs-Brock Web App Platform: Is it a Framework or is it an OS? (May 22, 2011)
• Francisco Tolmasky Objective-J and Cappuccino, with Francisco Tolmasky Interview by Zef Hemel State of Code (July 4, 2011)
• Chris Nelson Why it’s the late Cretaceous for server side MVC frameworks MysteryCoder (July 10, 2011)
• Kris Rasmussen The Luna Framework: Reactivity, Sync, and Co-Simulation (video presentation) Asana Blog Luna Presentation slides (July 27, 2011)
• Ian McCoy Web Development: We Are All Doing It Wrong Fine Shambles (July 29, 2011) Cogent.co
• Sridatta Thatipamala Why We Threw out All Our Code (And Why You Should Too) NowJS Blog (August 2, 2011)
• Wille Faler Tightly Coupled Web Frameworks Are Reaching the End of the Road, DZone (August 7, 2011)
• Nate Smith Announcing Racer, an Experimental Realtime Model Synchronization for Node.JS (August 17, 2011)
• Nicolas Petton & Göran Krampe Jtalk, the Smalltalk for Web developers (Slides are also written in Jtalk.) ESUG 2011 (European Smalltalk User Group Conference 2011) Edinburgh, Scotland, UK (August 29, 2011) About three weeks after this presentation, with the 0.9 release, JTalk was renamed Amber Smalltalk.
• Alon C. Programming: The dark side of Event Driven Programming This & That (September 9, 2011)
• Mikeal Rogers A new direction for web applications Future Aloof (September 27, 2011)
• Bernat Romagosa Pharo, will you take Amber for your lawful friend? A Smalltalk by the Seaside (September 26, 2011)
• Chris Nelson Web apps are dead, long live web apps Gaslight Software Blog (September 30, 2011)
• Alex MacCaw Asynchronous UIs – the future of web user interfaces (November 16, 2011)
•Ian McCoy Grand Unifying Theory Fine Shambles (November 24, 2011)
• Yehuda Katz Amber.js (formerly SproutCore 2.0) is now Ember.js (December 12, 2011)
• Jarod Ferguson Solving the upload progress bar problem–The History of Node.js Elegant Code blog (February 6, 2012) (Link to a video of a biographical talk by Ryan Dahl, to a Pheonix user group, Oct 11, 2011)
• James Burke Web dev with two turntables and a microphone</a Tagneto (January 30, 2012)
• Ian McCoy Stopping at the border Fine Shambles (February 23, 2012)
• Alex MacCaw Rails is just an API (March 4, 2012)
• Francis Hwang Should your web application be rich-client from day one? (March 5, 2012)
• Brad Gessler Don’t use Rails as “Just an API” (March 21, 2012)
• Brian Ford Is Node.js Better? (April 9, 2012) (From his talk given at JSConf 2012, April 3, 2012, Scottsdale, Arizona.)
• Isaac Z. Schlueter Re: @brixen’s “Is Node Better” (April 9, 2012)
• Paul B. Jensen Thoughts on Rails, Node, and the web apps of today (July 11, 2012)
• Daniel F Pupius The Rise of the Single Page Application (January 7, 2013)
• Spike Brehm @spikebrehm
Our First Node.js App: Backbone on the Client and Server AirBnB Engineering January 29, 2013) Hacker News commentary
• Zachary Vose The Web is Becoming Smalltalk (February 10, 2013)
• James Halliday @substack many things (March 14, 2013) (On libraries, modules and frameworks, and how frameworks make decisions the programmer is not aware of.)
“To be fair, there is a distinction here between frameworks that are sheer mudballs of compounded assumptions and frameworks that take the time to carefully factor out themselves into lots of tiny pieces. I think an important standard to judge frameworks is how many of the framework’s core abstractions can be used ala-carte without using the framework itself. If your framework melts away into an informal collection of modules that happen to work well together but can be easily repurposed by people who don’t use the framework, then you have built something very sublime.”
• Kyle Robinson Young @shamakry Using npm on the client side (May 27, 2013)
• Kyle Robinson Young Modules: The Right Way (October 21, 2013)
• Kyle Simpson @getify Silly Rabbit… Frameworks are for Prototypes getiblog (November 25 2013)
• Eric Normand React: Another Level of Indirection brLispCast (December 31, 2013)
(return to top of this post)
Links to Avi Bryant’s past and present blogs and the like:
• Avi Bryant (2012) avibryant.tumblr.com
• HREF Considered Harmful (2005 – 2010) avibryant.com (via Archive.org)
• HREF Considered Harmful (2003 – 2005) (Archive of blog hosted at Cincom)
• DabbleDB Blog (2004 – 2010) (Smallthought Systems, various developers posting, along with Avi Bryant)
• Twitter: https://twitter.com/avibryant
• Blocks: http://bl.ocks.org/avibryant
• Github: https://github.com/avibryant
• Bitbucket: Clamato: https://bitbucket.org/avibryant
Annotated links to Avi Bryant’s talks and posts
- Scala + Cascading = Scalding
Avi Bryant – Strata Conference + Hadoop World New York City (October 25, 2012)
• Presentation description & schedule: Scala + Cascading = Scalding,
The talk is listed in two Strata conference tracks:
• “Hadoop Tools and Technology”
• “Hadoop and Beyond“Avi Bryant’s abstract for “Scala + Cascading = Scalding”
Start on low heat with a base of Hadoop; map, then reduce. Flavor, to taste, with Scala’s concise, functional syntax and collections library. Simmer with some Pig bones: a tuple model and high-level join and aggregation operators. Mix in Cascading to hold everything together and boil until it’s very, very hot, and you get Scalding, a new API for MapReduce out of Twitter.Scalding is an open source Scala framework for concisely describing Hadoop MapReduce jobs. I started the project at Twitter as a way for ad server engineers to run simple queries on the ad logs, without needing to learn a specialized language like Pig, or dive too deeply into the guts of Hadoop. Since then, both the team and the framework have evolved, and it’s now used by 20 or so full-time data scientists, for all of their work: ads targeting, market insight, click-prediction, quality analysis, experiment analysis, and more. In this talk, I’ll walk you from its beginnings, expressing simple jobs like word count in a few lines of code, to its state-of-the-art present: word count is still a few lines of code, but so, for example, is PageRank.Scalding background
Twitter Engineering Blog: Scalding release and links
• Generating Recommendations with MapReduce and Scalding (March 2, 2012)
• Scalding 0.8.0 and Algebird (September 24, 2012)• Scalding: Powerful and Concise MapReduce Programming (video: 1 hour, 6 minutes)
By Oscar Boykin (@posco) & Argyris Zymnis, (@argyris) both from Twitter Engineering
San Francisco Scala User Group, San Francisco California (April 19, 2012)
- Smalltalk and Big Data
Avi Bryant – Keynote talk abstract (March 14, 2011)
Smalltalk Solutions (STS) 2011 Conference (March 13 – 16, 2011), Las Vegas, Nevada.
• Video: Smalltalk and Big Data (60 minutes)
• Slides: Smalltalk and Big Data
• Audio: available in more than one format via James RobertsonAvi Bryant on Seaside, and web frameworks generallyBryant explores implementing a Smalltalk DSL frontend for Hadoop MapReduce jobs, which became the foundation conceptual framework for Scalding, implemented with Scala on the JVM instead of Smalltalk.Before proceeding to the “big data” map-reduce part of the talk Avi discussed the future of his Seaside Smalltalk web application framework (and all other server-centric web app frameworks) in relation to the 2011 browser environment. The then-current (2010) browsers provide resources not available when Seaside was conceived in 2002:
- Seaside offers server-side stateful user-interface components (now, if state resides in the client, the server need not track state);
- Seaside offers unique callback IDs and this model of interaction architecture continues to grow in importance and has increasing acceptance as a general model. Cross-site request forgery (CSRF), for example, is much more difficult with session ID tokens in all transactions.
Avi Bryant speculated that if he were to build the next version of Seaside, or some Seaside-like framework in the current web and browser environment, he would include a JSON-builder with callbacks, and he would drop from the framework methods that have been demonstrated to be superfluous for recent (2010) browsers, in the same way that various historical conceptions were dropped during the original creation of Seaside. He would drop Seaside’s components and continuations, Seaside’s server-side canvas API, while retaining Seaside’s callback and session token ID model in a re-designed framework.
Selected quotations from “Smalltalk and Big Data”
- “I may be saying Seaside is obsolete, but I have also said at a Rails conference, ‘Rails is obsolete‘ and I have also said at a Django conference, ‘Django is obsolete‘, because the fact is that none of the web frameworks out there right now are built with this architecture in mind. We still have an opportunity with Smalltalk to get out ahead, and still be in the forefront of web framework design. But we can’t rest on our laurels with what was the forefront of web framework design five or ten years ago.”
- “I believe that there is a big difference between something that was designed for one architecture and has been adapted to a new one, and something that has been designed for a new architecture from the ground up. I believe if we tried [for] a clean-slate redesign of Seaside, not trying to keep any kind of support for the older architecture, that we would find new and better ways of doing things than when we were trying to graft something new onto what’s already there [in Seaside].”
- Burn the diskpacks
- “A lot of the original design of Seaside was about throwing away what people believed to be important constraints: ‘You have to have shared nothing, you have to have clean URLs’ “
- “Seaside was an exercise in: ‘What if we throw away those constraints? What do we do, where to we end up, where do we go?’.”
- “What I’m saying is it’s time to throw away the constraints of ‘we’re going to generate the HTML [on the server], we’re going to have a server-side component hierarchy.’ “
- “If we throw away those constraints, then where do we end up? I’m pretty sure it will end up in some place better than if we keep those constraints.”
• Big data at Twitter background: Twitter’s analytics processes are described in a couple of presentations by Kevin Weil, who talks about Hadoop and MapReduce, and how (as of October 2010) 12 terabytes of daily data (tweets, time-stamps, followers, retweets, logs, etc.) were analyzed.
NoSQL at Twitter InfoQ (video, 56 minutes) Strange Loop 2010, St. Louis (October 14, 2010)
Hadoop and Pig at Twitter (video, 26 minutes) (slides) Kevin Weil, Hadoop Summit 2010 (June 29, 2010)
• Link to all Smalltalk Solutions (STS) 2011 presentations: videos and slides, and speaker bios
• Comment by Joachim Tuchel: Seaside without Continuations? (April 27, 2011)
- Rails is obsolete (but so’s everything else)
Avi Bryant – Presented at GoGaRuCo 2010 – Golden Gate Ruby Conference (September 18, 2010) San Francisco, California.
• Avi Bryant’s description of the talk: http://gogaruco.com/schedule.html#bryant
• Video: Rails is Obsolete (But So’s Everything Else) (36 minutes) (video released April 11, 2011)Selected quotations from “Rails is obsolete (but so’s everything else)”
- “Web apps do not work the way they did in 2004.”
- “How should we change the 2004 design [of Rails] to work for 2010 web apps?”
- “It comes down to whether you care that there’s a bunch of stuff in your framework that is no longer useful, and whether you believe that that actively gets in the way. I believe that it is important to strip things down to the core of what you’re actually using…and I think that that probably leads to more interesting developments. If you take the step of throwing away views and controllers, then there’s a sort of a vacuum there that you are going be to actively working to fill. If you say, ‘Well, OK, I won’t use that part of the framework,’ the potentials for surprising elegance are probably less.”
- “My esthetics are towards pushing the envelope…”
- “I would argue that the most interesting new designs happen when you go back and throw away the old design decisions because your constraints have changed, and I so am loathe to continue to take on the constraints that I am hoping we are are rid of.”
- Django is obsolete (but so is everything else)
Avi Bryant – Keynote at DjangoCon 2009 Django Conference (September 8, 2009) Portland, Oregon.
• Video: Django is obsolete (but so is everything else) (48 minutes)
• Summary and commentary (and links to others’ DjangoCon presentation slides) by Ted Leung: DjangoCon 2009
- Failure: An Illustrated Guide
Avi Bryant – Presented at FutureRuby (July 12, 2009) Toronto, Canada.
• Video: Failure: An Illustrated Guide InfoQ (22 minutes)
Interview with Avi Bryant by Werner Schuster at FutureRuby (July 9-12, 2009) Toronto, Canada.
- Speeding Ducks
Avi Bryant – Presented at JAOO Sydney 2009 , Sydney Australia (May 5-8, 2009) and also at JAOO Brisbane 2009 (May 11-14, 2009). (Organized by the since-renamed “GotoCon”)
• Slides: Speeding Ducks
Avi’s description of the talk, in the talk’s abstract:
Also at these two Australian conferences he gave a talk entitled
• 1,001 Iterations: Product Design, Illustrated, an earlier version of the talk on describing developing Trendly that was presented two months later at the Future Ruby Conference, called “Failure: An Illustrated Guide”.
- MagLev: Ruby That Scales
Avi Bryant (Dabble DB) with Bob Walker (GemStone Systems, Inc.),
O’Reilly RailsConf 2008, Portland Oregon (May 30, 2008).
A pre-alpha demo and description of the architecture of Gemstone’s Maglev, a full-stack Ruby implementation, demonstrated running on a modified Gemstone/S Smalltalk VM and object-oriented database (with some Ruby-specific bytecodes; the bytecode is JITted to native code before execution). Maglev has an integrated distributed object cache, and integrated object data persistence storage. This pre-alpha version of Maglev was sufficient to run the WEBrick HTTP server library.
• Video: MagLev presentation at RailsConf 2008 (17 minutes)Background to the Gemstone Maglev OODB effort described at Avi’s blog HREF Considered Harmful:
• Ruby and other gems (March 8, 2008)
• Maglev (May 28, 2008)
• MagLev recap (June 1, 2008).Related Maglev commentary:
• Bob Walker (of Gemstone) MagLev: Gemstone builds Ruby runtime based on Smalltalk VM InfoQ Interview by Werner Shuster (April 30, 2008)
• Chad Fowler: Maglev (June 5, 2008)
• Glenn Vanderburg: Maglev (June 5, 2008)
• Antonio Cangiano MagLev rocks and the planning of the next Ruby shootout Zen and the Art of Programming
(May 31, 2008)
• Charles Nutter Maglev Headius (June 1, 2008)
• Obie Fernandez MagLev is Gemstone/S for Ruby, Huge News (May 30, 2008)Subsequent presentations and interviews about Gemstone and Maglev development by Gemstone personnel
• James Foster The Seaside Heresy (at OTUG) (video 1 hour 42 min.) Object Technology User Group, Minneapolis/St. Paul Minnesota &nsp; (September 16, 2008)
• Bob Walker, Allan Ottis: Ruby Persistence in MagLev (video, 54 minutes)
(November 6, 2008) RubyConf 2008, Orlando, Florida, USA
• Martin McClure, GemStone Systems: Maglev (video, 60 minutes) (December 1, 2008)
A question and answer session at Pivotal Labs describing the implementation process and thinking behind of Maglev.
• Martin McClure, Gemstone Systems: presentation on building Ruby in Smalltalk for the Maglev project.
• Video: Ruby in Smalltalk at ESUG 2009 (43 minutes)
• Summary text ESUG 2009 conference report by Niall Ross (see pages 2 – 6 of this 57 page PDF) ESUG 2009 conference (European Smalltalk User Group). Brest, France   (August 28 – September 4, 2009)
• James Foster Introduction to Gemstone (six videos, totalling less than an hour) Smalltalks 2009 Conference, Buenos Aires, Argentina. (November 19, 20, 21, 2009)
• Maglev Version 1.0.0 was released October 31, 2011. (Gratuitous github 1.0.0 release commit on github.)
• Monty Williams interview by Werner Shuster NoSQL OODB with Smalltalk-based Ruby VM: MagLev 1.0 Released InfoQ (November 15, 2011)
- Turning the Tables: Moving Beyond Relational Storage
Avi Bryant, MeshU 2008 (May 20, 2008) (The day before Mesh Conference) Toronto, Ontario Canada .
A workshop tour through design considerations for non-relational database choices, and when to consider memory store, flat files, object database, or a data service like Amazon’s SimpleDB and Google’s AppEngine. (He also gave an early demo of his work to create a pre-alpha version of Gemstone’s MagLev at this conference, which also demo’d at RailsConf 2008 a week later.)
- Ruby and other gems (Data persistence using Gemstone)
Avi Bryant’s blog – HREF Considered Harmful (March 08, 2008)
- Chatting with Avi Bryant
Interview of Avi Bryant by Fabio Akita (Posted December 15 and 22, 2007).
A text interview outlining the conceptions used for Seaside, the implementation of DabbleDB, the process for bootstrapping the DabbleDB startup, and commentary comparing Ruby and Smalltalk.
• Part 1 – Chatting with Avi Bryant
• Part 2 – Chatting with Avi Bryant
An excerpted quotation of Avi Bryant from the interview:
“The main thing I think WebObjects did right was to focus on the state and behavior of the application, rather than on the mechanics of URLs and HTML and HTTP requests. So rather than worry about what a field was named, you would just say ‘this field is bound to this instance variable in my model’, and rather than worry about what URL a link went to, you’d say ‘this link triggers this method on my page object’. Transitioning between pages was done by constructing a new page object and setting it up directly in Java, rather than constructing a URL that was going to be parsed to build a page. That directness and general style is something that very heavily influenced Seaside.”
- FLOSS Weekly 21: Avi Bryant on Seaside Podcast (November 23, 2007)
Host interviewers Randal Schwartz and Leo Laporte.
This Week in Tech (TWiT). (Running time 59 minutes.)
Avi describes his progression to primarily using Smalltalk and his co-creation of the Seaside Smalltalk web application framework in the pre-Ruby on Rails era. His explorations were inspired by Apple’s pre-Java Objective-C version of WebObjects, with an initial exploration in creating the Ruby web application framework Iowa (which was abandoned in favor of relying on Smalltalk and developing Seaside). Seaside, as an open source project was taken over by the larger community during the course of version 2. Interestingly, in passing, Avi admits he’s not much of an implementer of tests while programming in Smalltalk.
• Podcast: http://twit.tv/floss21
- Web Heresies: The Seaside Framework
Avi Bryant’s talk at OSCON 2006 – O’Reilly Open Source Convention, Portland Oregon (July 27, 2006)
An introduction to Seaside and its architecture, describing its original design using closures and shared state. Surveying the architecture of Canvas, Component, Callback and Continuation models: how that Seaside’s HTML generation API frees the programmer from the presentation layer details; how continuations allow freedom from thinking about state-machine workflow, enabling application modularity.
• Avi’s OSCON 2006 report to the Seaside email list. The presentation slides were not published.
• Commentary on Web Heresies session, by Kevin Yank, Sitepoint (July 28, 2006).
- An Introduction to DabbleDB
OSCON 2006 O’Reilly Radar: The Executive Briefing (July 25, 2006)
Brief presentation on DabbleDB. Sharing the stage with with presentations by other startups:
Tim O’Reilly, O’Reilly Media, Inc.
Kevin Cochrane, VP of Web Content Management, Alfresco;
Adrian Holovaty, Editor of editorial innovations, Django
Dave Rosenberg, Founder and CEO, Mulesource
Javier Soltero, CEO, Hyperic
Mark Spencer, President, Digium
Jeff Waugh, Consultant, Gnome/Waugh Partners
Scott Yara, President and Co-Founder, Greenplum
- Web application frameworks in Smalltalk and Common Lisp
Avi Bryant (Seaside) & Marc Battyani (Fractal Framework)
Vrije Universiteit Brussel, Belgium (April 26, 2005)
• Avi Bryant’s abstract of of the talk:
On the web, abstraction is a dirty word. The dominant paradigms and philosophies of web development — CGI, Servlets, Server Pages, REST– provide only a thin wrapper around the low-level details of HTTP, and encourage you to use the rough stones of the transport protocol as the direct building blocks of your application. Web developers by and large reject any further abstraction in the way that assembly hackers once rejected structured programming: it’s too inflexible, uses too many resources, and above all, it doesn’t let you see what’s *really* going on. As a result, web applications suffer the same problems now that assembly language programs did years ago; they’re fragile, verbose, difficult to maintain and ill-suited to reuse.
That’s not to say that better abstractions aren’t available. The Lisp and Scheme communities have been working on them for years. Paul Graham’s ViaWeb pioneered the use of closures, not query parameters, to capture application state in each link. Christian Quiennec showed how to use first class continuations to invert the flow of control of HTTP and put the server back in the driver’s seat. Macro packages like htmlgen bring HTML into the language itself, opening up much more than a template system can provide. Meanwhile, object-oriented packages like WebObjects have demonstrated how to decompose the web page into a tree of stateful, interacting objects, allowing a finer granularity of development.
Seaside combines these ideas and others with the rich development environment of Smalltalk to provide a stable, complete, and innovative web application platform. This talk will introduce Seaside, and will focus in particular on the ways in which these abstractions can be leveraged to enable reuse: how to use closures, continuations, and intelligent HTML generation to destroy the intra- and inter-page coupling that is holding web development back.
• Marc Battyani on Lisp Fractal Framework. Slides: Fractal Framework (Powerpoint)
and an article with more detail: A Framework for Automatic Web Application Generation
• Comment on the topic by Bill Clementson: Using a Lisp Web Server Behind Apache
- Making Object Relational Data First Class
Smalltalk Solutions (STS) 2004, Seattle, Washington, USA. (May 4, 2004)
Abstract of the talk (alternative link):
ROE, the Relational Object Expression library, models relational queries as first class Smalltalk expressions. This has several advantages over using SQL strings directly:
• queries can be built using familiar Smalltalk syntax and without worrying about binding and escaping data
• queries can be easily composed, so that a complex query can be built up over several methods, none of which know any details about the others
• queries look like ordered collections of tuples, but with most operations other than #do: defined lazily; for example, #copyFrom:to: won’t pull in any data, but will simply produce a new query with an extra clause
• queries maintain a rich set of metadata, so that, for example, the columns of any query can be automatically grouped by table and its rows mapped properly into objects.
ROE can either be used directly as a better interface to relational databases, or as a new foundation for object/relational mapping tools. Currently, it has only been tested using PostgreSQL and Squeak Smalltalk, but it could easily be ported to other platforms.
- HREF Considered Harmful: Structured Web Development with Seaside Tutorial
Avi Bryant & Julian Fitzell Smalltalk Solutions (STS) 2003, Toronto, Ontario, Canada. (July 14, 2003)
• Workshop description• Avi Bryant comments on WebObjects by Apple/NeXT (in Java) and IOWA (in Ruby) (August 18, 2003)
- Initial pubic description of Seaside (2002) (Beta4.com archived page)
Avi Bryant & Julian Fitzel begin supporting Seaside and marketing development of applications using Seaside. Links to initial documentation with architecture and design overview for developers using Seaside.
- Explicit Programming
1st International Conference on Aspect-Oriented Software Development
Enschede, Netherlands (April 23 – 26, 2002)
Presentation of the paper Explicit Programming by Avi Bryant, Andrew Catton, Kris De Volder, Gail C. Murphy (all then of University of British Columbia, Vancouver BC, Canada). Published in AOSD 2002 – Proceedings of the 1st International Conference on Aspect-Oriented Software Development. ACM – Association for Computing Machinery.
“Many design concepts can be expressed only indirectly in source code. When this occurs, a single concept at design results in a verbose amount of code that is scattered across the system structure. In this paper, we present explicit programming, an approach that enables a developer to introduce new vocabulary into the source to capture a design concept explicitly. … We believe explicit programming provides a useful engineering point, balancing modularization and separation in (at least) two cases.”
• Explicit Programming (PDF)
- Explicit Programming: Improving the Design Vocabulary of Your Program
Avi Bryant, Andrew Catton, Kris De Volder and Gail Murphy
(Demonstration) OOPSLA 2001 (Tampa Florida)
• Explicit Programming demo proposal (PDF) (via Kris De Volder)