“Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object- oriented or otherwise.” – Ward Cunningham
Ward Cunningham and other thought leaders within the IT industry define the term “Technical Debt” as restricted to code for the application under development and potentially avoidable. There are many other technical aspects of a software project, however, that could be defined and managed under the same categorization. Does the analogy of technical debt not work in the same way when pushing off, for example, a much needed infrastructure library, operating system or integrated development environment upgrade? As we continue to develop code in a less effective way against an older library, are we not effectively paying interest on this unpaid debt? Over the years, through my experience and discussions with many practitioners, I’ve found that the definition is more useful to both developers and managers when it encompasses all debt of a technical nature. As a result, I would like to propose a broadening of the definition of the term, but let’s first explore its origins.
Origins of the term…
In the early 90’s, when the software industry was relatively young and immature, there was a strong need to classify ‘clean up’ work so that non-technical folks could easily understand this type of work. I believe that over time the original definition by Ward Cunningham has served its purpose well, however respected thought leaders in our industry have stayed true to constraining the definition:
“You have a piece of functionality that you need to add to your system. You see two ways to do it, one is quick to do but is messy – you are sure that it will make further changes harder in the future. The other results in a cleaner design, but will take longer to put in place.” – Martin Fowler
“Technical debt is bad design. Good design is modular: It has each discernible bit of design in a single place, not spread all over. […]” – Ron Jeffries
Perhaps being consistent with the original definition is partly out of respect for the man who originally coined the term (the same person that coined the term ‘wiki’). Throughout my career, however, I have found that practitioners vary more than one would expect on a term so core to our work in software development.
What practitioners say…
To help better understand this disparity, I conducted a brief survey amongst a handful of colleagues with a variety of roles and experience levels here at Intelliware. At a high level, I found that:
- There was agreement on the criticality of both developers and managers being involved in overseeing technical debt and how to manage it.
- There was a difference of opinion on the avoidability of technical debt and the level of visibility to stakeholders.
- A recurring theme of ‘passage of time’ came up as a cause of technical debt. A few specific examples include:
- the potential to upgrade to Java 1.8 to leverage the productivity benefits of lambda expressions
- teams coding internationalization logic in hibernate 3 are losing out on efficiencies of later versions of the library
- when an opportunity to update a development environment to improve development efficiencies is not taken to capitalize on short term goals
As I reflected further on ‘passage of time’ as a cause of Technical Debt, I realized that in an Agile environment, the term cannot remain true to its original definition. For example, application code written in perfect accord with requirements would inherently create debt without the application code even being touched. One of the core tenets of Agile is to not anticipate or code for future requirements, so we work hard to keep the code simple and aligned with the domain in question. However, the instant a new requirement is introduced, the code becomes incrementally misaligned and naturally has a debt that needs to be ‘repaid’.
Reflecting even further, I realized that in my own career I was effectively the sole owner of all such debt on a team of over 50 people, giving me a rather unique perspective on the subject.
My experiences with it…
About 15 years ago I joined a Model Driven Architecture (MDA), Rational Unified Process (RUP) project of about 8 people that kicked off the elaboration phase and was moving forward at a comfortable pace, being proactive, and properly dealing with Technical Debt as it came up. We had a stable team size and the appropriate amount of time to make big decisions on our platform and development environment. Now I wasn’t in management this early in my career, but in retrospect it seems that when we hit the Construction phase of the project, there was no time allocated to anything but feature work. Furthermore, with the expectation that things had to go according to plan, we didn’t have much ability to pace ourselves, nor deal with Technical Debt, as we quickly ramped the team up to over 50 people. As the only developer that also dealt with all items relating to team productivity, including DevOps (before the term was coined), I saw first-hand what can result from the mismanagement of Technical Debt and found myself alone in dealing with the vast majority of it.
Having recently read the “The Phoenix Project”1, I was a bit like Brent. I was the only person on the team with deep technical skills that worked across all sub-teams; analysists, quality assurance, developers, user interface developers. When there was an issue, I was often the go-to guy that could fix things quickly. This gave me a unique perspective on the productivity deficiencies of the team, backlogging everything as it came up. Management knew the positive impact I had on the project as I imparted my knowledge back into the product and scripts through double-loop learning2. It was clear to me through this and other experiences in my career that project managers were never doing enough to keep their ear to the realities at the ground level and adjusting accordingly. Business priorities took precedence, to a fault.
To help convince management of the importance of my work in this area, I estimated scope and savings against all of the 100+ items on my backlog and tallied up well over $250k/year in savings (excluding additional qualitative and compounded savings). This worked out to be over 10% savings per year and although this seems reasonable on a concentrated area such as team efficiency, it was about 15 years ago and unfortunately it was one of those projects where management found themselves tied too tightly to the plan to be able to react appropriately.
Since then, the term ‘DevOps’ helps to appropriately resource the majority of the work I was doing back then, but from a management methodology perspective, it seems we are still short in addressing the root cause of the problem, and the fact that we don’t seem to have consensus on a unifying term to describe all inefficiencies of a technical nature certainly doesn’t help.
How a redefinition of the term can help…
In the project I just described, management wasn’t actively searching for ways to improve team efficiencies – they were solely focused on feature development for the application. If there was an efficiency available that could increase developer productivity, it was materialized always by chance, discipline or overtime. There was never any active/deliberate discovery exercise allocated for such work and it’s never actively prioritized and managed. In the absence of a unifying term for these inefficiencies, that naturally come up in the course of development and maintenance of software, it’s no surprise that there is a gap in modern project management literature in how to manage it.
Taking a look at the state of the art of software development today, would our current management methodologies naturally prioritize efficiencies to be had, above story work, in the absence of a unifying term? In the absence of DevOps, would it naturally arise as a major area of concern after a few sprints as it should? Is there an expectation that a sustainable pace is attainable without the continued prioritization of technical debt? Certainly retrospectives help to some extent, but how do they help with respect to slightly larger initiatives that may not be so obvious? How do they facilitate researching such options?
Steve Freemen had a compelling argument that Technical Debt is more analogous to an ‘unhedged call option’3 and others have challenged the term, so maybe ‘Technical Debt’ isn’t the right categorization of such issues. It is clear though that if we do proceed to use it, it is more useful to both developers and managers to broaden it to include all debt of a technical nature – not just that residing in the application code. From what I have seen on recent projects, in speaking with many colleagues and with the small survey I conducted, it already seems to naturally resonate with practitioners in this way. It seems to be the best term we have for the purpose of categorizing all productivity deficiencies faced by a team of a technical nature and will allow us to move discourse forward in how such work is managed.
1 Kim, Gene, Kevin Behr, and George Spafford. The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win. N.p.: n.p., n.d. Print. Coincidentally, the internal project I was on was also code named ‘Phoenix’.
2 “Double-loop learning is contrasted with ‘single-loop learning’: the repeated attempt at the same problem, with no variation of method and without ever questioning the goal” — https://en.wikipedia.org/wiki/Double-loop_learning
Technical Debt Definition Ron Jeffries: http://ronjeffries.com/xprog/blog/technical-debt/
Technical Debt Definition Martin Fowler: http://martinfowler.com/bliki/TechnicalDebt.html
Technical Debt Definition (Ward Cunningham): http://c2.com/doc/oopsla92.html