March 9th, 2011 started with Clay Shirky's keynote. Clay studies the effect of the Internet on society and the economy, and his talk was by far the best keynote of the conference. The main point I got out of this is that you can't always solve a novel problem by using an existing mindset.
Keynote: Clay Shirky
Clay started his talk using an example of the first functioning steamboat, built by John Fitch:
...the thing is, that was his fourth attempt. This was his first:
Note, that what Fitch did was just project the old mechanics (the oars) on the problem. New technology seems to follow this trend. The CMSs, for example, of the 90s were like Fitch's first steamboat. They used old mechanics to try to solve a new problem. They basically just served the content from a central location for consumption, rather than than take advantage of the type of two-way communication which the Internet makes possible.
Clay showed some stats for a page on Wikipedia relating to Dr. Who. The stats show a curve, where one person contributed the vast majority of content to this article (thousands of edits), while just a few people contributed hundred of edits, a few more people ten or so edits, and thousands of people contributed one edit. It turns out this is a common pattern, especially when looking at open-source projects.
The take-away is that in the new, participatory model of the technology, there is no one model of the user. We need to design for the power-user as well as the "average" user.
Some other great points Clay made:
- Behavior as a first class object. We judge others on their behavior, we judge ourselves in a context. We think we can judge others by watching them in a few situations based on their behavior. This comes out in how we design for users. We think of them as "dumb" when they don't "get it", but when we don't "get it" we blame the software.
- People will behave well when the incentives to do so exceed when you don't. Think of ebay reputations...
- Design against the bad behavior. Delicious decided not to adjudicate spam. It's just designed so spam isn't noticed. As a consequence of the design, SPAM tends to sink to the bottom and nobody pays it mind. If nobody notices SPAM, does it exist?
- Single loop organizations fix problems; double loop organizations fix problems and the situation that caused the problem.
- Self modifying organizations. Nomic game, proposed stack overflow sites in Area 51. Most of these sites don't launch; they are self-vetting.
- Traditional CMS were the oars that rowed the boats. Now CMS can help 'nodes' on the internet be a potential meeting place for people and be self-correcting. In a sense the nodes on the Internet have become their own beings.
- Open source community is a bit different than the business community and has a chance of leading the way. The open source community has the idea of benevolent dictator who doesn't intervene much, allowing contributors to "get a room" in order to work out differences. But when develpers are at loggerheads, the dictator comes in and makes a decision.
Truly Clay's talk was fantastic. Probably the most visionary talk I've ever heard on technology.
New Directions for Drupal Documentation: Jennifer Hodgdon, Ariane Khachatourians
I've been interested in contributing back to an open-source project for a while, but having two kids and a busy schedule, it's hard to find the time. I figured I'd go to this presentation, because I use the docs all the time, and thought this would be a way I could jump in and start contributing. Jennifer and Ariane just took over as leads of the docs team, so they laid out their immediate vision, admitting that they haven't had time to think long-term goals over much. The presentation did give me the bit of impetous to start assigning myself a few documentation issues in the queue.
Automated Testing & Drupal: Steven Merrill
Steven did a quick walk-through on how to write some tests for a Drupal module using SimpleTest. I'm glad I had a little experience with doing that beforehand or I would have been completely lost, as SimpleTest for Drupal is a bit more complicated than a unit test. SimpleTest runs on a virtual, clean version of Drupal in a fixture that uses prefixed tables to simulate an additional Drupal install.
Steven did show some differences between running SimpleTest in Drupal6 and 7. For one, in D7, you no longer have to figure out a module's dependencies manually. Now, SimpleTest figures out the dependencies for you (using the dependencies declared in the .info file, I assume).
Another great change, is the ability to dump values to the screen or to the log using the
debug() call. In D6, it was very hard to debug tests and made writing complicated tests virtually impossible.
Steven mentioned a fascinating module called Selenium to SimpleTest, which translates Selenium IDE PHP output to Selenium 1 tests. Again, interesting, but pretty useless to me, because I write my tests in Selenium 2, don't use the IDE, and write my tests in Java. After the talk, I spoke with some folks at the TreeHouse Agency, Steven's firm, and was shocked to find that they mostly use IDE output for their tests. I've gotten to the point myself, where I always write my own Selenium code from scratch—I just like the control it gives me.
Steven spoke next about Cucumber, a Behavior Driven Development (BBD) tool. I am really in to Test Driven Development (TDD) lately, but the BBD takes it one step further by forcing us to think at a broader organizational level, instead of getting caught in the trees. So, Cucumber basically allows:
- Business requirements to be written in plain English (following a certain standard).
- Requirements to get automatically translated into testing code.
- Requirements to get translated into documentation.
So, you've killed three birds with one stone. I'd love to take a look at this when I get a chance.
Failure to Launch: Drupal Performance Tuning: Kenny Silanskas
On top of knowing his shit, Kenny is a really funny guy. I'd say he's probably the second funniest guy at this conference, after Jared Spool.
He talked a bit about using reverse proxies to improve performance and showed how it allows http requests not to "stick" to the web server. He used a very impressive visual to demo this, by having people in the audience shoot nerf guns at a sheild he constructed of cardboard. The nerf darts, like http requests, stuck to his shield (the web server). Then he demonstrated reverse proxies by bringing out a shield in the shape of Cookie Monster, which he constructed out of blue spandex and PVC. Out of the many jokes he made, he mentioned that they looked at him a little oddly in Home Depot when he purchased latex and PVC.Here are some other ways to make Drupal sites faster:
- Pressflow is "...is a distribution of Drupal with integrated performance, scalability, availability, and testing enhancements." In addition, it gives you a new performance page, apart from the built-in Drupal performance page, which is a bit sparse.
- Varnish is a generic, reverse proxy that helps make your sites much quicker by not blocking as Apache does.
- syslog is better than dblog because it doesn't touch the db.
- Stop using Drupal's statistics module, at least not on large pages. It writes to the database and negatively affects performance.
- Views does some evil things. For example, the node style loads the whole node.
- Use Views cache.
- Always purge sessions-table, logs regularly.
- Set session cached settings in php.ini: something more sane, like three weeks.
- 404s do a total re-bootstrap for Drupal. Avoid them at all costs. Here's an interesting module to address this: Fast 404.
- Verify age of caching in headers by refreshing the page and using a header tool, like Live Headers.
- Seige: a utility to help you test your website under duress.
The first thing I noticed when I walked into this talk, was that the presenter is 15 years old. That's right. He's not only a major contributor to the Drupal project, he's a tenth-grader, and an award-winning composer. WTF? I can't even tell you publicly what I was doing at 15.
Dimity showed us how to help solve the age-old Drupal problem: configuration management. Often distributing a working Drupal site can be challenging, because the CMS/framework is so modular. A particular configuration could depend on multiple modules and multiple settings. The Features module makes it easy to define a "type" of Drupal site by packaging up entities like module dependencies, views, CCK fields etc. What Features does is package your configuration up into a module, which can be extended, installed, and updated just like any other Drupal module.
Profiler is a level higher than Features. It allows you to package up Drupal core and exported features into an install profile, which you can then use to setup a complex Drupal site in a minimal amount of time.
Drush Make is an extension to Drush that allows you to package up entire Drupal sites with features, profiles, all using a simple text ("make") file containing dependencies.
Indeed, Dmitry succeeded in demonstrating the utility of these three tools working in concert, and I admire his balls in demoing this live. There are so many things that can go wrong.
To be honest, I didn't totally follow him, as he went pretty fast, and I think he could have done a better job of explaining the differences between these three tools and how they compliment eachother. Also, at work I use Drupal in a very different way than most people do, so I am not sure how this would help me in my configuration management woes. For example, there are configurations I do that I'm not sure would be captured by using these three tools in concert. But it would be worth exploring in order to solve my configuration management challenges.
Advanced Drush Moshe Weitzman, Owen Barton, Mark Sonnabaum
This session was not actually planned in advance, though it drew hundreds of folks eager to learn new Drush tricks. The original session during this slot was canceled, and this one was given instead.
If you haven't used Drush, it's great. It's an example of what makes Drupal an awesome CMS to work with, because so many open-source tools are already built for it. With Drush (DRupalSHell) you do a lot of the things that would require you to set things via the clunky UI, or that would take multiple steps at the command line. For example:
drush dl views
..downloads and extracts the latest version of Views module in the correct place.
...fires up the MYSQL command line, within the context of a Drupal install bootstrap.
As I expected, I learned a whole lot more, through I am not entirely sure what features are just in the newer 3x Drush--that I will have to research further:
- a drushrc.php files placed in the same directory as the Drush script sets global Drush configurations, much like .vimrc can be used to customize vim instances. Anyway, you can set Drush aliases here, which make it so you don't have to be in a Drupal directory to issue commands to that Drupal instance. There are many other things you can set in drushrc, but I have yet to find really good documentation about what those things are. This is definitely something worth exploring. Once I do get to exploring this, I'll probably go here: http://www.lullabot.com/articles/new-features-drush-3 and http://developmentseed.org/blog/2010/mar/10/drush-30-more-powerful-flexible-and-magical.
- sql-sync: This command does what a sqldump command does, but without you having to specify the host, username, database, password etc. on the command line, since the settings.php file has all that information.
- The -sanitize flag when dumping the db, masks private info, like emails and the like.
hook_drush_sql_sync_sanitize allows you to intercept this process and do what you will with it.
- drush policy. This feature allows an admin to restrict what a dev can do with Drush. For example, you probably don't want drush touching a production site. I plan to look more at this: http://emspace.com.au/article/drush-aliases-primer-live-dev-syncing, when I have time.
- Lastly, the presenters showed what can be done to extend Drush and script with it. The cool thing here is that since Drupal is bootstrapped, you can do anything in your Drush script that you could do in devel's execute PHP box.
So much was presented here, and it will take some time experimenting to see how I can make my Drupal development more efficient. The challenge, of course, is finding the time.