I’ve just returned from QCon San Francisco yesterday and it was one of the best conference I’ve been. There were so many interesting sessions with time conflicts, I had a hard time deciding which ones to go to. All the interesting names were there: eBay, Netflix, Facebook, twitter, just to name a few. The hottest topic of the conference – NoSQL – judging by how packed some of the rooms were for these sessions.
There’s too much information to include in just 1 blog so I’m listing 3 of the key architectural lessons I got from the sessions with a promise for more to come later.
Understand Your Data
I attended several sessions about systems of scale including eBay, Netflix and Facebook. Data appeared as one of the central themes in all. At scale, your data and cache has to be scaled horizontally in addition to the app. In order to do this effectively, the data has to be well modelled and understood.
Many of the solutions employ eventual consistency to achieve performance at scale. This can be implemented either at the application level or within the database. NoSQL databases in the main use this consistency model, trading consistency for higher availability. With this type of model I think some of the key questions become “what data can tolerate being temporarily inconsistent?” and “when the application is unable to resolve the inconsistencies, what data would you need?”
Interestingly, almost all of the strategies at scale model their data using key-value pairs, which reduces the importance of a schema at the database level. However, it really means that the schema still exists in the application and a poor schema means poor joins and access patterns, and difficult validation at the application level.
It is just too easy nowadays to ignore the database and data with the many layers of abstractions we have but ironically, at scale it is not just something the DBA has to think about.
At scale, not only is it necessary to scale horizontally but also to easily and automatically provision or decommission an app server. In the cloud, this capability is required since machines by definition come and go and your infrastructure has to be able to cope with change very quickly. Having no state in services and the app makes migration or synchronization unnecessary. Extending this principle to the application level, beans should have clear separation of state and behaviour so that they can be easily composed into complex services.
Instrumentation And Logging
Instrumentation and logging are very common in many of sessions. They are not only really useful for analytics but also for debugging. With very large systems that tend to have a lot of asynchronous interaction, especially if they exist in the cloud, logs are the only means to reason about bugs or behaviour.
Given the amount of data that the instrumentation and logs will generate, logging has to be consistent and machine readable so that any significant event in the application can be detected.