mkeblog2023-12-30T18:51:39+00:00http://mkelandis.github.ioM. Landisblog@hintersphere.comManaging Technical Debt2015-02-07T00:00:00+00:00http://mkelandis.github.io/software%20engineering/2015/02/07/managing-technical-debt
<p>In a previous article, I introduced <strong><a href="/software%20engineering/2014/03/06/tech-debt/">technical debt as a <em>system</em> with a positive feedback loop</a></strong>. In this one, I discuss using <strong>Agile Practices</strong> to tackle not only the debt, but also the <strong>Technical Debt System</strong> in order to minimize and break the positive feedback loop. Let’s talk about the how and the when, on the following fronts:</p>
<ul>
<li>Business Pressure, Inadequate Review, Definition of Done, Delayed Refactoring, Fear</li>
</ul>
<p><img src="/assets/posts/2015-02-06-managing-tech-debt/techdebt-attack-state.png" alt="image" /></p>
<p>###How</p>
<p>####Labels and Filters</p>
<p>Hopefully you’re using an <strong>issue tracking</strong> system. Track technical debt just as you would any other task or bug. Use labels and filters so that you can bring up a Technical Debt Dashboard. Simply by tracking technical debt we have removed <strong>fear</strong> and promoted <strong>transparency</strong> in order to have better conversations that help alleviate <strong>business pressure</strong>. Now we’re in a good position to tackle it.</p>
<p>####Definition of Done</p>
<p>You have a <strong>Definition of Done</strong> right? Having an adequate definition of done is a key component in constraining the accumulation of technical debt. Your definition of done could include but not be limited to:</p>
<ul>
<li>Acceptance Criteria for Story is met.</li>
<li>Unit Tests that exercise your changes.</li>
<li>Integration Tests that exercise your changes.</li>
<li>All tests pass.</li>
<li>80-95% branch & line test coverage (decide what thresholds make sense for <em>your</em> team).</li>
<li>Has been code reviewed by N team members.</li>
<li>Static Analysis and Code Formatting rules.</li>
</ul>
<p>####Continuous Integration</p>
<p>Exercise as much of the <strong>Definition of Done</strong> as possible in your <strong>Continuous Integration</strong> builds, and have them fail when these standards are not met. This removes <strong>fear of refactoring</strong>, thereby limiting the introduction of new technical debt.</p>
<p>Run CI builds on your topic branches such that code which doesn’t pass CI doesn’t get merged into an integration branch. Extra bonus - running CI on topic branches avoids the practice of shaming developers who break integration.</p>
<p>####Backlog Grooming</p>
<p>Prioritize and point your technical debt dashboard during grooming sessions and work with the <strong>Product Owner</strong> to order these items in your regular backlog. Ask clarifying questions like “<em>What is the business value of refactoring this?</em>” and “<em>What is the cost of not refactoring it?</em>”</p>
<p>Keep in mind that the <strong>Product Owner</strong> should be responsible for ordering the backlog, including issues of technical debt. It may seem obvious, but a key success factor for an agile team is iterating on business value, always pulling in the highest value from the top of the backlog. Cherry picking other tasks is analagous to <strong>stealing from the project</strong>.</p>
<p>On the other hand, determining the business value of technical debt relative to delivering stories can be tricky. Remember that the cost of technical debt includes <strong><a href="/software%20engineering/2014/03/06/tech-debt/#time-to-deliver">Time to Deliver, Code Smell and Bugs</a></strong></p>
<p>####Sprint Planning</p>
<p>Pulling technical debt points into your sprint becomes trivial if it’s part of the ordered backlog. However, if ordering technical debt isn’t working, establish a <strong>policy</strong>. Examples:</p>
<ul>
<li>Our team can do 20 points per Sprint and we pull in 3 points of technical debt during each sprint planning session.</li>
<li>Every third sprint we do 10 points of technical debt.</li>
</ul>
<p>####Sprint Retros</p>
<p>Agile Values include <strong>Transparency</strong>. Dedicate time during Sprint Retros to raise up issues of new technical debt, and ensure that these issues are comprehended in your Issue Tracking system.</p>
<p>Do you have a <strong>policy</strong> for tackling tech debt? …is it working? …does it need to be changed?</p>
<p>Accumulation of tech debt could point to an <strong><a href="/software%20engineering/2014/03/06/tech-debt/#inadequate-review--dod">inadequate definition of done</a></strong>. Is your definition of done up to date? Does it need to be re-evaluated?</p>
<p>###When?</p>
<p>####Right now</p>
<p>On the ground, in the trenches… If <strong>Delayed Refactoring</strong> can be avoided, take some extra time and work through your technical debt right now. Be a steward of the code and leave it better than you found it. If you can’t address it now, then make sure you track it as an issue.</p>
<p>Pull technical debt points into your sprint during <strong>Sprint Planning</strong>.</p>
<p>####During Code Reviews</p>
<p>Code Reviews can and should be <strong>iterative and conversational</strong>. Budget enough time in your process and <em>expect</em> to make changes based on peer feedback. Make an effort to identify technical debt during code reviews and take the time to address those issues.</p>
<p>####When it’s Blocking You</p>
<p>When technical debt blocks you from completing your work, that’s probably a red flag that it’s time to tackle it. You may be able to put in a work-around but it’s likely going to just keep blocking you in the future. Resist the temptation to adjust the Definition of Done (lowering test coverage thresholds for example) in order to move on.</p>
<p>####Dedicated Sprints</p>
<p>More often than we might like to admit, technical debt is accumulated more during the last bit of development, close to release time. That is when we’re most likely to experience elevated <strong><a href="/software%20engineering/2014/03/06/tech-debt/#business-pressure">Business Pressure</a></strong>. We’re more likely to sacrifice the <strong>definition of done</strong> in order to get a release out the door.</p>
<p>Although this particular risk can be mitigated by practicing <strong>Continuous Delivery</strong>, that technique is for another time, another blog.</p>
<p>Once that big release goes out the door, there will likely be a period of relaxed <strong>business pressure</strong>. This is a really good time to pay back technical debt in a big way. It may make sense for your team to dedicate an entire sprint theme to technical debt, with buy-in from the Product Owner.</p>
<p>###Agile Practices</p>
<p>Hopefully you can see how <strong>Agile Practices</strong> address Business Pressure, Inadequate Review, Definition of Done and Delayed Refactoring in order to attack the <strong><a href="/software%20engineering/2014/03/06/tech-debt/#technical-debt-has-a-feedback-loop">system of technical debt</a></strong>. Using these techniques you can prevent and tackle technical debt on your projects.</p>
A System of Technical Debt2014-03-06T00:00:00+00:00http://mkelandis.github.io/software%20engineering/2014/03/06/tech-debt
<p>In this article we introduce an eight state <em>system</em> of technical debt, and analyze it’s behavior.</p>
<h3 id="what-is-it">What is it?</h3>
<p>According to an <a href="http://www.computer.org/portal/web/computingnow/1211/whatsnew/software">article by Frank Buschman</a>, the term “<strong>Technical Debt</strong>” was coined by Ward Cunningham and is defined as:</p>
<p><em>…a metaphor for the trade-off between writing clean code at higher cost and delayed delivery, and writing messy code cheap and fast at the cost of higher maintenance efforts once it’s shipped.</em></p>
<p>While that is an excellent definition, it isn’t always obvious and dramatic. Even mindful, vigilant teams who follow agile practices including peer reviews, continuous integration, and a thoughtful definition of done (DoD), still find themselves accruing tech debt.</p>
<p>Tech Debt comes in many forms, including but not limited to <strong>insufficient test coverage</strong>, <strong>code quality problems</strong>, <strong>inadequate documentation</strong>, <strong>out-of-date dependencies</strong>, and <strong>failure</strong> of code <strong>to meet project standards</strong>. Examples of code quality problems might typically include messy code, duplication, unnecessary complexity and the oft tempting re-invention of the wheel.</p>
<h3 id="why-does-this-happen">Why does this happen?</h3>
<p>If you review the <a href="http://en.wikipedia.org/wiki/Technical_debt">wikipedia entry on this topic</a>, you might notice that many items listed as <em>causes</em> of tech debt appeared to actually be <em>examples</em> of tech debt. This is confusing until it becomes clear that…</p>
<h3 id="technical-debt-has-a-feedback-loop">Technical Debt has a Feedback Loop</h3>
<p>Technical debt actually begets more tech debt over time, and it deserves it’s own state diagram:</p>
<p><img src="/assets/posts/2014-03-06-managing-tech-debt/techdebt-state.png" alt="image" /></p>
<p>Let’s walk through different states of tech debt and attempt to understand the flow.</p>
<h4 id="business-pressure">Business Pressure</h4>
<p>Sometimes we can hear our product owners channeling Larry the Cable Guy, drawling “Git-R-Done!”. Customers care about business value, delivered in a timely fashion, according to the budget. They are often less interested in the details of code coverage, code quality metrics, multiple peer reviews or whether or not we’re several versions behind the latest version of [insert your favorite framework here].</p>
<p>This business pressure can cause a team to compromise review and/or standards established by it’s definition of done.</p>
<h4 id="inadequate-review--dod">Inadequate Review / DoD</h4>
<p>Inadequate review can be caused by business pressure or an inadequate definition of done.</p>
<p>Delivered code may deviate from the technical design, contain wheel re-inventions, include duplications or may be incomplete in terms of <strong>done</strong>. For example, the code might include unit test coverage and still be missing integration tests.</p>
<p>When the definition of done is missing or inadequate, different standards may be applied when a task is completed. (The DoD helps alleviate this by providing a checklist for authors and reviewers who can then practice adequate peer reviews against a standard.)</p>
<h4 id="delayed-refactoring">Delayed Refactoring</h4>
<p>Business pressure and inadequate review lead to delayed <a href="http://en.wikipedia.org/wiki/Code_refactoring">refactoring</a>.</p>
<p>The art of writing computer code is a process of continual restructuring and iterative rework. At some point we must accept a level of completeness in order to deliver the solution. At that time, there could still be more work that needs to be tackled… refactoring is delayed. We may be aware of this delayed refactoring situation at the time when a decision is made, or we might not be aware of it.</p>
<p>While delaying things now results in short-term gains, the long-term implications of not performing them (code coverage as a glaring example) leads to tech debt.</p>
<h4 id="technical-debt">Technical Debt</h4>
<p>Delayed refactoring leads to tech debt.</p>
<p>The decision or non-decision to delay refactoring means we now have tech debt. Often, we have incurred a future cost in the system that will be more painful to pay off down the road then if it had been addressed right now. This tech debt could be in the form of:</p>
<ul>
<li><strong>insufficient test coverage</strong></li>
<li><strong>code quality problems</strong></li>
<li><strong>inadequate documentation</strong></li>
<li><strong>out-of-date dependencies</strong></li>
<li><strong>failure</strong> of code <strong>to meet project standards</strong></li>
</ul>
<h4 id="fear">Fear</h4>
<p>Tech debt leads to fear.</p>
<p>You are on Degobah and your ship just sank to the bottom of a bog. it’s stinky, dark and foggy. Something large and silent just slithered a few feet from where you stand on an unbalanced tangle of tree roots. A voice seemingly from nowhere (which sounds oddly like Miss Piggy with throat congestion) asserts “Technical debt leads to fear, fear leads to delayed refactoring, delayed refactoring leads to technical debt, technical debt leads to”… we are now caught inside the feedback loop of <strong>Technical Debt</strong>, <strong>Delayed Refactoring</strong> and <strong>Fear</strong>.</p>
<h5 id="scenario---insufficient-test-coverage">Scenario - insufficient test coverage:</h5>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Alice: "I really want to refactor this code, I need to use it but it's smelly and I think there might be a bug in one of the bounds conditions"
Bob: "If you refactor that, what else is going to break?"
Alice: "I don't know."
Bob: "Don't touch it, make a copy"
</code></pre></div></div>
<p>Congratulations, we just created more technical debt. The only way out? You guessed it - “Suffering”.</p>
<h4 id="time-to-deliver">Time to Deliver</h4>
<p>Tech debt eventually demands to be paid off, which increases time to deliver.</p>
<p>Ironically, by compromising in the immediate interest of time we have actually made it more difficult, and therefore longer to deliver a future feature because at best we will need to meet the definition of done <em>and</em> perform the refactoring that was delayed just now. In the worst case, as more technical debt piles up, time to deliver will continue to increase until eventually managers and customers will wonder why we can’t seem to get anything done on time.</p>
<h4 id="code-smell">Code Smell</h4>
<p>As compromises are made in the interest of delayed refactoring, code quality suffers and developers may also start to notice strange patterns or <a href="http://en.wikipedia.org/wiki/Code_smell">smells</a> developing in the codebase. These smells are a flag that something needs to be refactored. They can also lead to future bugs and increased time to deliver, as co-workers struggle to decipher what the code actually does.</p>
<h4 id="bugs">Bugs</h4>
<p>If your team cuts corners on unit or integration test coverage, it risks delivering code that hasn’t been exercised, especially at the bounds. That means you don’t really know (and cannot prove) if it works or not, and it’s likely to have bugs.</p>
<p>Another less obvious path to bugs happens when dependent library upgrades are neglected. For example, if our dependencies fall out of date, we may miss a security fix. If we are eight versions behind the current stable, supported version and code coverage is low, an upgrade cannot be performed without incurring significant new, unplanned cost in terms of time and effort in addition to dollars.</p>
<h3 id="end-states">End States</h3>
<p>When not placed in check, these last three end states represent suffering and pain in our system. In the worst case this results in an unmaintainable product, and destroyed business value. Hopefully the importance of adequate review and high standards for definition of done are becoming clear. The feedback loop in the system makes it imperative that tech debt is tackled iteratively to prevent it from snowballing. In the next article we explore agile practices and strategies for dealing with tech debt.</p>