QCon San Francisco 2011: Thursday

Keynote: Things I Wish I’d Known

Rod Johnson springsource

The keynote address by Rod Johnson, founder of the Spring framework, was focused mostly on his experiences in growing the Spring, and later springsource, business.

  • Don’t Start Coding – When starting a new enterprise, programmers often have a tendency to start coding far too early in the process. Don’t sink your time into code until it is necessary, and then only as much as is required to advanced your business to the next required stage. As a founder, your time is extremely valuable, and you can’t afford to spend 70 hours a week coding.
  • Don’t Skimp on the Lawyer – Smart, technically savvy people can generally handle most aspects of running a business – finances, market analysis, accounting, etc. However, one aspect that they don’t often grasp is the law. Get legal advice. Get it early. Consult with your lawyer often. You can never be too careful.
  • Running an Open-Source Team Across Borders is Hard – Spring started as an open source project which was then commercialized. When Rod tried to include the top contributors in his enterprise, he ran into a series of legal and taxation issues caused by his employees being in different countries than the one he was in (Springsource was a UK corporation).
  • Business is Rewarding – Rod now spends most of his time away from the code – growing the business, writing books, and giving talks (like this one). Getting to this point involved a lot of hard work, but it was worth it, and proved to be a very rewarding experience.

Slides: http://qconsf.com/dl/qcon-sanfran-2011/slides/RodJohnson_ThingsIWishIdKnown.pdf

Open APIs: 2011 State of the Market

John Musser – Programmable Web

John Musser of Programmable Web, an online directory of web apis, gave a rundown of the evolution of online open APIs to their current state. A few trends noticed:

  • Protocol: While programmatic web sees APIs using both SOAP and REST, the number of REST APIs as grown exponentially compared to SOAP
  • Data Formats: XML, JSON, RSS, ATOM, etc. The number of APIs supporting JSON has grown significantly, as well as the number of APIs which do not support XML.
  • Authentication: HTTP Auth was heavily used in the past, but oath is quickly becomming the standard.
  • Money: The business model and pricing structure of the APIs have become much more complex in recent years. From simple free, pay per developer, etc, to more complex volume-based pricing.
  • Business Model: The api has become the core business product, instead of simply an extension of the product
  • Versioning: There is no standard solution for api versioning, some services use URL versions (ie. service/api/v1), some use version numbers as http header values.

See: google api discovery service, swagger.net

Slides: http://qconsf.com/dl/qcon-sanfran-2011/slides/JohnMusser_OpenAPIs2011StateOfTheMarket.pdf

Developing for Cloud with Java EE7

Adam Leftik – Oracle

This presentation focused on the new features coming to Java EE7 to support cloud services and Platform as a Service (PaaS).

Java EE Roles now support the idea of tennancy, allowing java processes to be temporary residents of an application server, to support Infrastructure as a Service (IaaS) environments. The benefits of IaaS and elastically sized server clusters are …

The latest version of Glassfish is to be the reference implementation for the IaaS and PaaS features in JEE. The new features include elastic scaling and instant configuration of load balancers, infrastructure, and Derby databases for an application’s needs.

The new features in JEE7 are not limited to Iaas and Paas domains. There is also:

  • improved CDI for dependency injection
  • modernization of JMS
  • pruning EJB CMP (breaking backwards compatibility) to remove burden from container vendors
  • web profile
  • JPA 2.1, EJB 3.2, JAX-RS 2.0, Web Socket

Oracle is also aiming to be completely transparent with the JSR process for these new improvements.

Slides: http://qconsf.com/dl/qcon-sanfran-2011/slides/TylerJewell_JavaEE7DevelopingForTheCloud.pdf

Keeping Movies Running Amid Thunderstorms!

Siddharth Anand – Netflix

This session discussed the architectural choices made by Netflix to support the massive userbase and traffic needs spawned by their move to internet streaming as a delivery platform for their content.

Prior to the introduction of the streaming service, Netflix used a fairly standard Data Center architecture/deployment model, consisting of a Hardware Load Balancer supporting a wide cluster of tomcat/apache servers, with Oracle and MySQL databases serving content. Scaling this deployment involved improving the hardware that ran the services and databases. It was a constant struggle to squeeze every last ounce of performance out of the hardware stack.

This architecture needed to change when Netflix switched their business model to content streaming. In the new Cloud Architecture, each Netflix service is run as an application in a cluster. These applications generally fall into one of three categories:


  • sit behind an amazon elastic load-balancer
  • handle all public traffic from the web
  • do not communicate with other edge services
  • access mid-tier services


  • support business functionality
  • internal
  • communicate with other mid-tier and backend services


  • storage – cassandra ring
  • amazon s3, sqs
  • memcache

Amazon cloudwatch is used to monitor the cpu usage of servers in the cluster and automatically expand the cluster if needed.

This architecture is more horizontally scalable as more clusters can quickly be brought up to provide more instances of services when load is increased. Fast deployment is also enabled, as new servers can have the latest version of production code. Selective, configurable load balancing can be used to ease traffic onto the new service, and bring it back if things start to fail.

This architecture does require more network hops than the previous scenario, and has more moving parts, but the benefits appear to outweigh the new costs.

Thundering Herd

A common problem faced in the Netflix cloud is keeping the service running in a Thundering Herd situation. Thundering Herd occurs when, for one reason or another, requests from the Netflix edge services begin to timeout, those requests combined with the retries for these requests will overload servers in the cluster. To compensate, you can bring up more servers, but how many? If you were to double the size of the cluster, the original requests and their (possibly multiple) retries will still be too much for the cluster to handle. A drastic scale increase, followed by a gradual retreat, is needed.

According to Sid (and Facebook) one of the key success factors to maintaining a fast development cycle is too minimize the cost of mistakes (not eliminating mistakes, just minimizing their cost). One of the ways they do this is through the use of the Simian Army – a series of programs which produce or simulate failures on the Netflix cloud:

  • Chaos Monkey– simulates hard failure in cluster auto-scale groupl>
    • Latency Monkey – simulates soft failures
    • Limits Monkey – checks if going over capacity on amazon cluster/services
    • Conformity & Janitor Monkey – looks for unused resources and frees them

    Slides: http://qconsf.com/dl/qcon-sanfran-2011/slides/SiddharthAnand_KeepingMoviesRunningAmidThunderstorms.pdf

Technical Debt – Why to care and what it means to your business

Felipe Rubim – Ci&T

This session was a fairly standard refresher on what technical debt is, the decisions that introduce debt into your project, and it’s consequences. The causes of technical debt were analyzed, including pragmatic motivations (choosing frameworks for expediency) and sloppy coding. The concept of paying interest on technical debt was explained, and the signs you can watch out for that indicate that interest on your debt is piling up. For a more complete discussion on technical debt, see my post from last year’s Toronto Agile Tour, or Martin Fowler’s article.Slides: http://qconsf.com/dl/qcon-sanfran-2011/slides/FelipeRubim_TechnicalDebtWhyToCareAndWhatItMeansToYourBusiness.pdf

Refactoring to Pattern

Joshua Kerievsky – Industrial Logic

This session provided advice regarding the use (or overuse) of design patterns in software refactoring.

In general, judiciousness, or blind fanaticism, when using design patterns when coding or designing/architecting systems, can lead to gross inefficiency and poor design. So, conservation is required.

Refactoring to Patterns is a catalog of examples in which overly complicated code is refactored to simpler patters. The afterward of this text observes that many of these refactoring patterns are merely the application of the same set of algebras. The operations in these refactoring algebras are cataloged in his online catalog. Some examples include:

  • graceful retreat
  • gradual cutover
  • preparing for change
    parallel change

Trick for refactoring a delete:

  • instead of using a safe delete, or deleting and fixing the string of compiler errors, comment out the body of the method to be deleted, and use Refactor Inline to replace all instances with an empty line.Slides: http://qconsf.com/dl/qcon-sanfran-2011/slides/JoshuaKerievsky_RefactoringToPatterns.pdf
It's only fair to share...
Share on Facebook
Tweet about this on Twitter
Share on LinkedIn

1 thought on “QCon San Francisco 2011: Thursday

  1. Pingback: QCon San Francisco 2011 | i-proving.com

Leave a Reply