QCon Day 1: Musicians, Google, Open Space & Netflix

So after all that fun during the first weekend (and about which I’ll blog separately), now it's the time to finally shift our attention to the reason for which we're in San Francisco in the first place; InfoQ's highly rated internal software conference QCon. Since my previous visit to QCon was such an impressive experience, my expectations were high. After dropping off our rental car at Alamo’s, we strolled along Market Street to the new venue, the Hyatt Regency at Embarcadero Center, a longer walk than I expected.

As usual, the keynote was preceded by an introduction of each of the tracks hosted by the corresponding track owners. However, I couldn’t resist the notion that some of them were really not feeling comfortable being on stage. In fact, it all sounded like some kind of obligation they couldn't get away from. Fortunately, the keynote speaker, Rich Hickey, was so much better suited for this. He elaborated on a great analogy between software developers and professional musicians. He explained how musicians always start with ideas about short melodies, cadences and rhythms and use that to compose a harmonious song. In our profession that would be the equivalent of creating small autonomous components and use those to 'compose' a system 'in harmony'. If those components end up to be too complex, then you should really work hard to take them apart. Just accept that a large system will eventually end up as too complex to grasp completely. Instead, focus on the small parts first and then look at the bigger picture.

 

But he even went further with that. He explained us that instruments are always created for experienced musicians, simply because of the fact that you're only a novice for a short time. So why do we still optimize our code for beginners? Just like musicians practice every day, rather than making our ‘instruments’ (our tools) simpler, we should practice hard as well. Definitely a very inspiring talk, in particular due to the many references to the origins of electronic music (which, as a former fan of Jean-Michel Jarre, really resonated with me).

The 2nd session of the day was done by a Netflix architect, Jeremy Edberg, and dealt with how they managed to get such a scalable and reliable solution, something you would take for granted when using their services. The big 'secret' is that they always design their systems with the assumption that something will go wrong eventually. He mostly showed us how they build custom tools to monitor all their environments, but definitely impressed with the vast amounts of energy they’ve invested in making their systems so resilient against failures. And now that I think of it, I noticed that all those companies start to create their own tools after they’ve grown to a certain scale. Definitely something to remember the next time you’re looking for an off-the-shelve tool to solve your problem.

The slot just before lunch was reserved for an Open Space on architecture. As somebody who really spends a great deal of my time on agile and architecture, I can tell you it was a great experience. First I joined a discussion on agile architecture and how to get the business to understand how important bringing down technical debt is, and why we need time to work on architectural changes. I was glad to hear that this challenge is a universal problem in our profession (and not only mine :-)). The main take away for me was that you should really try to avoid running in stealth mode and spend more energy explaining the business the cost of technical debt.

 

I couldn't resist proposing a discussion on Event Sourcing and using it to build occasionally connected system. It was fun to explain a group of experienced architects how and why we used events as a unit of synchronization. The big question was whether I would choose this architecture style again, provided similar requirements. But after some contemplating, my answer is 'most definitely yes'. However, I did emphasize the fact that I underestimated the complexity it introduces, even though I've been talking about this topic for several years now.

After having selected the wrong talk in an already bad slot (the speaker didn't get to his point until after 30 minutes), I decided to check out that new noBackend hype. It was mostly about how a solution that has virtually no backed isn't so susceptible to NSA hacks. By the way, did you hear about this story that some AT&T engineers found a network splitter in their server room, and discovered it was used by the NSA to route all data to their servers for analysis? The speaker, Parker Higgens, was working for a foundation that is supposed to help companies dealing with this NSA issue, but I was mainly surprised that they could even talk about all this in public.

A talk I was looking forward to was the one where Rachel Laycock would explain us how to adapt your architecture to facilitate continuous delivery. She discussed a lot of architectural and design principles, in particular Conway's Law, but what I missed is how she would approach teaching existing teams about this principles in such a way they'd really get it. Don't get me wrong though, all of what she says makes sense and aligns perfectly with my own ideas, but it's the challenge convincing other people about it that's the difficult part of our job. I’m sure going to have a chat with her somewhere this week.

One session I did get a lot out of it is the one on how Google has setup their developer workflow. In our current project we've recently started using feature branches to move towards a more stable trunk. But just imagine the surprise when I learned Google has about 10000 developers all checking in on the trunk about 20 times a minute. But after learning their aggressive testing strategy and how they've optimized their automated build-and-test pipeline I've gained a renewed goal to improve ours as well. They don’t even have a traditional QA group anymore. Developers are held responsible and they facilitate the testing efforts by injecting a test engineering professional into each team. In fact, test evangelism is a quintessential aspect of the developer workflow. A good example is the Testing On The Toilet practice, a single page of new ideas and experiences that developers are encouraged to read at the toilet. How's that about dedication…

 

After serving all those beers, I was surprised to see how many attendees appeared at the post-day keynote. On the other hand, that proves once again that the QCon audience generally has a lot of passion. Well, the fact that we’re still here at 19:30 proves ours as well…