What Is Technical Debt?

Technical Debt (sometimes called code debt) is a concept that was first talked about by Ward Cunningham way back in 1992 and has since been covered by people like Steve McConnell and Martin Fowler to varying levels.  Rather than repeating everything they've said I'm going to try and I explain it in my own words.  To me technical debt is best thought of as the technical issues and problems we have that slow us down and prevent us from adding as much new functionality to our software as we would like, in the time we have available.  This is typically seen in teams that have a decreasing velocity or noticed when changes that once took half an hour to make now take a couple of hours.

Now, when most people are initially presented with this concept they will immediately think of bugs and it's easy to understand why.  Bugs take time to fix and are an obvious cause for a drop in the time available for adding new functionality.  But bugs aren't the only form of technical debt.

Technical Debt can come through a number of channels, sometimes deliberate but more often unintentional:

1. Bugs.  Obviously.  Bugs have to be fixed.  Time is required to fix them.  However a large number of bugs decreases a developers confidence in the code.  When developers change the code they will either take more time to make sure they haven't broken anything else as a result, or will ignore the other problems and in the process add more bugs.

2. Lack of Automated Testing.  Without automated testing developers are unlikely to be sure that they're changes are correct.  They (or testers on the team) will spend extra time to make sure that the changes haven't broken anything obvious, thus slowing down the team.  Unfortunately, they are unlikely to do a full regression test nor are they likely to try multiple methods to break their code, which means it's likely that the number of bugs in the system will increase.

3. Poor Architecture.  Architectural costs are hard to measure, but impact the team dramatically.  Poor architecture is typically evidenced in hard to test code, where code changes need to be repeated in multiple locations, where different approaches to the same problem are seen in the same application, where things just don't make sense, and where developers openly joke about the crap code that they have to deal with.

4. No Coding Standards.  Coding standards exists to help increase the readability of code.  People should be able to look at code and see a common coding style that anyone on the team is familiar with. Code that is hard to read takes longer to understand and is harder to make a non-buggy change to.  It's also common to end up with multiple coding styles apparent in the same method or file, making it even harder for people in the future to understand it.

5. Highly Complex Code.  Code with a high cyclomatic complexity takes time to understand.  Making a non-breaking change is often even harder.  If effort is taken to refactor the code then that also will take time.

The Problem

"So why is this a problem?" I hear you ask.  Lets think about it.  Let's assume we start a brand new project. No code. No frameworks to deal with. Nothing.  A blank slate.  Now when the project starts out we have the ability to add as much new functionality to the project as we can within an iteration.  And we do, however in that first iteration our team ends up creating bugs.  It's just the nature of the beast.  We're all human, and we all make mistakes.  But since we've only just started we're not that worried about those bugs at the moment.  We'll deal with them later.

Unfortunately our developers are all individuals and each has their own coding styles and standards. And because they haven't really done it before they don't think about unit testing or loosely coupled architectures.  It's a very common scenario.

So the first iteration concludes, and in that first iteration our team delivers a large amount of new functionality and we're really, really pleased about it.  Congratulations all round.  Unfortunately and unseen we've also accrued some technical debt but it's not really a concern at this point, right?

So our next iteration begins.  We now start making changes to existing code to extend on functionality we delivered in iteration 1.  Unfortunately some of those bugs we weren't worried about are stopping us from completing our new work and our team has to spend time fixing those bugs as well as making their changes.  Things they thought would be fairly quick to do end up taking more time than expected and we also find that developers changing each others code are having issues understanding what it's doing and working with other people's coding styles.

At the end of the iteration the team still delivers a good chunk of functionality, but it's not as much as they were hoping for.  The reason - it just took longer than we expected.  While we're not as happy as we were at the end of the first iteration, it's still a good delivery and it's not much below what we were hoping for.

The next iteration we see the same patterns of behaviour as in iteration 2. And this time our team delivers less functionality than ever before and we're not happy.  The developers aren't sure why things are taking longer, they just are.  Maybe it's the tools, maybe it's the BA's fault for not writing better specs, whatever the case, it's not developers.  They're trying just as hard as they were at the start of the project, so it can't be that.

What is the root cause of the problem?  As you probably guessed, it's the technical debt.  As our technical debt increases we have less capacity for adding new functionality resulting in frustration for both our team and our customer.

Left unchecked we will eventually get to a point where we can't add any new code at all because we don't have any spare capacity left.  We end up with a team spending all their time understanding code and fixing bugs just trying to keep the damn thing running.

Here's the problem in graphical form:

image

Solutions - Option 1

OK, so we have a problem.  Our team is stuck in the mire of endless bug fixing.  How do we get out of it?

Here's what most people will do: hire more staff!

This is the worst thing you can do.  It's like getting an overdraft on a loan you can't pay back.  What's worse is that any new staff you get will take time to get up to speed and will undoubtedly make more mistakes initially than anyone else because they haven't got the experience the others have in dealing with how the application works.  Plus, because the root cause of the problem has not been dealt with our code debt will just continue to increase until we're out looking for more staff again.

It looks like this:

 image

Solutions - Option 2

Option 2 is a more tantalising one.  Start again! We can scrap our current code base and start fresh, learning from the mistakes of the past.

This is a very tempting option for a number of reasons.  First it means we can write off all that technical debt we accumulated. We can upgrade our development tools to whatever the latest and greatest release is. We can get excited about doing things differently and we can tell ourselves that because we learnt so much from our previous mistakes that this time will be sooo much better.

This sounds great in theory, but we're forgetting something.  The application itself and the bade code isn't the problem.  A crappy code base and a poor application is a symptom.  The problem lies with our people and the way we work.

If we start fresh and don't deal with the fundamental issues then we're just going to repeat the problem of increasing technical debt, only we'll be doing it with different tools.  Given enough time we'll be right back where we are now, with an unmaintainable application and an unhappy team, only it'll be worse this time because we'll remember that this was a rewrite and weren't things supposed to be different this time 'round?

Solutions - Option 3

OK. So hiring more staff is out.  And starting again is probably not the best idea in the world.  What do we do?

Fix the fundamental problem.  It's a hard thing to do.  We have to press pause on our development work and spend the time to clear some of our debt.  And this is not an easy thing to do - especially if we've got an ingrained and established pattern of debt accrual and a demanding customer or overriding business issues.  Plus you have to somehow convince the business that there's value in doing it and you have to convince the developers to change their ways.  My only suggestion is to try and confront the issue head-on and deal with them openly and honestly.  Call it a learning experience and use the charts above to back up your arguments for change.  You might just get lucky and buy yourself the time you need to save the situation.

 

If you do there here are some suggested starting points for clearing the debt.

Start by agreeing to some coding standards as a team - and follow those standards! No lip service, no saying one thing and doing another.  Make sure that you do code reviews or pair programming and that your team polices each others work.  Use tools like StyleCop and FxCop to assist in your efforts.

Next try to fix some bugs.  Don't try and fix all of them, it'll take too long.  But fix some of the big and ugly ones, the major pain points.  And do so using the newly agreed to coding standards.

Then try to add some automated testing - add unit tests, add functional tests, do anything that will give you some confidence that changes you make don't break something else.  Do what you can to break dependencies between classes, to improve your architecture and to simplify your code.

Anything you can do to make your code more maintainable is going to help in reducing your technical debt.

Finally, after an agreed period of time start taking on new development work again, but this time don't take on as much as you can fit.  Take on some work but keep some capacity for further clearing of technical debt.  It will take time, but done right, you'll break the debt cycle and get back to doing what you do best - delivering great software for your customers.

Oh, and don't be too concerned about clearing all of your debt.  Every team carries some form of technical debt, the difference is that the best team's carry as little as possible.