Reversing technical debt with the Cloud

The following article covers best practices for managing and reversing technical debt through cloud services' prudent use and operation.

Technical debt is a metaphor coined by Ward Cunningham to deal with the cost of deciding to compromise software development to meet short-term business needs. An example? In the case of financial debt, you take out a loan to purchase something you want or need immediately, with the promise of repayment later. If you fail to repay, you suffer the consequences. Similarly, if software deficiencies are not addressed promptly (‘repayment’), maintaining and adding new features to the software becomes exponentially more complex (‘debt interest’). This applies to all companies operating locally or in the cloud. However, just as financial debt needs to be managed responsibly, technical debt must also be handled meticulously. A vital advantage of the cloud is managing technical debt effectively.

What are the causes of technical debt?

Many factors contribute to technical debt. Starting with the obvious, poor practices and technology decisions cause technical debt. Examples include poorly written software, inadequate code reviews and testing, lack of documentation and standards, inflexible software design, and inappropriate technology choices. Technical debt also results from a lack of IT skills and capabilities and frequent staff turnover.

Business and technology teams interpret product requirements differently, resulting in technical debt. Business requirements focus on the functional end-user experience. Technology requirements, on the other hand, concentrate on non-functional attributes such as resilience, performance, maintainability, and security. These are often not prioritized by companies. To meet deadlines, IT teams usually take shortcuts (introducing technical debt) to accommodate these non-functional features. Changes in customer preferences, the environment, competition, geopolitics, the economy, and regulations make it difficult to predict future business models. Therefore, creating future-proof software is a challenge. Moreover, technology is evolving rapidly, making tomorrow's technology choices obsolete. Hackers constantly find new ways to exploit software vulnerabilities, which puts us in a constant state of technical debt.

Best practices to manage technical debt

Unfortunately, technical debt is inevitable. How do you manage it? Left to its own devices, it would make it more difficult to make software changes, which in turn cripples speed and inflates delivery costs. On the other hand, if companies spend too much time pursuing perfection, delivery will be affected. It is worth considering that technical debt is not necessarily a bad thing. As long as technology debt is deliberate but prudent, with a well-defined implementation path to address shortfalls ('pay off') in the future, it is a win-win for both business and technology teams. Here are some best practices for managing technology debt through cloud services.

Recognize 'real' technology debt and make it visible. Technology debt can be misused to justify investment. It is usually used to justify improving internal elements of IT systems rather than adding new features available to the user. Investment in technical debt should, therefore, be linked to business outcomes. The business impacts, such as poor customer experience, lost revenue, delayed product releases, longer resolution times, and reduced employee productivity, should quantify debt. Periodic maintenance activities such as backups, third-party updates, and security patches are not part of technical debt. These are simply good operational practices.

Technical debt is not just a technology issue. The decision to incur and pay off technical debt should involve business and technology teams. Stepping back from customer needs and prioritizing the most impactful minimum viable product (MVP) features can help business teams reduce technical debt. Business teams can reduce requirements turnover by clearly defining when a software feature is 'finished and ready' and not promising stakeholders too much. Similarly, IT teams should be open and realistic about the time and resources needed to create new features or pay down debt.

Use cloud services. While 'lifting and shifting' local workloads to the cloud is an excellent option for moving quickly to the cloud, it does not solve the technical debt problem. In this case, local debt moves to the cloud. To maximize the benefits of cloud deployment and manage technical debt, companies need to think carefully and align their local processes to the cloud. Deploying applications requires what AWS calls 'undifferentiated heavy lifting', such as managing hosting, bandwidth, contracts, capacity, operations and coordination between large teams. Improper management of these areas can have negative consequences and thus generate technical debt. Cloud platforms provide the ready-made building blocks needed to develop and maintain software. This allows teams to focus on business value and innovation rather than internal IT elements. Infrastructure provisioning, code deployment, testing, threat detection, monitoring and alerting, and disaster recovery can be fully automated in the cloud. Self-recovery functions automatically shut down unused infrastructure or scale up in response to demand. Many cloud services, such as serverless computing and purpose-built databases, are fully managed with built-in features for high availability and resilience. IT teams do not need to allocate resources to manage routine tasks.

Apply agile practices and a cloud-native approach. Cloud computing lends itself well to agile programming and is based on experimentation, flexibility, and incremental delivery principles. Cloud native technologies support rapid and frequent application changes without affecting service delivery. An agile development approach favors speed over perfection and improves business-technology collaboration. Both new features and technical debt are visible in a unified 'product backlog.' Teams then prioritize the backlog and deliver incremental value in short sprints.

Mature development practices. Code reviews, establishing straightforward design and development standards, proper documentation, and automated testing are techniques for reducing technical debt. In addition to coding practices, modern applications use loosely coupled technologies and focus on event-driven and serverless components. Modern applications also use automated infrastructure and operational and security tools to improve the reliability and consistency of software deployment. Reusable and proven software components are an excellent way to speed delivery and reduce technical debt.

Training/skills development. Deploying new technologies requires equipping staff with the right skills to make informed decisions and implement these technologies effectively. Consider the case of cloud technology: a recent AWS Global Digital Skills survey found that 87% of employers said that investing in employee digital skills training helped their organizations achieve their digital transformation goals faster. From an employee perspective, more than 80% of respondents reported improved productivity and higher job satisfaction. These factors help companies achieve a faster investment return and reduce technical debt.

Measuring technical debt. Several approaches have been proposed to measure technical debt, such as cyclomatic complexity, code coverage, square assessment, code violations, and even software bug counting. While these provide some quantitative measures of code quality, they do not measure other dimensions, such as the frequency of changes made to the code. Code that has low quality but is rarely changed should rank low when prioritizing a solution. Technical debt is not always apparent until you look at it. The year 2000 is a classic example of when, in software, the 'year' was written using two digits ('75' instead of '1975') in the 1960s and 1970s. By the turn of the century, it had become the principal debt of all companies. Technical debt is meaningful when expressed regarding business impact, such as poor customer experience, lost revenue, longer release cycles and employee productivity. This simplifies investment and prioritization decisions.

Final thoughts

Technical debt is real and cannot be avoided. This is due to both controllable and uncontrollable factors. The risk of ignoring this is high as it can result in longer time to market, lost revenue and a negative impact on team morale. Business and technology teams need to address technical debt together. This should be visible and prioritised. Indeed, cloud technologies offer effective ways to transform technical debt into technical prosperity.

Reference literature

Agile Alliance, D. W. (n.d.). Project Management and Technical Debt. Retrieved from https://www.agilealliance.org/project-management-and-technical-debt/

Fowler, M. (2009, October 14). TechnicalDebtQuadrant. Retrieved from https://martinfowler.com/bliki/TechnicalDebtQuadrant.html

Fowler, M. (2019, May 21). TechnicalDebt. Retrieved from https://martinfowler.com/bliki/TechnicalDebt.html

Schwartz, M. (2020, December 16). The CIO-CFO Conversation: Technical Debt-An Apt Term? Retrieved from https://aws.amazon.com/blogs/enterprise-strategy/the-cio-cfo-conversation-technical-debt-an-apt-term/

Whelan, J.-L. L. (n.d.). Introduction to the Technical Debt Concept. Retrieved from https://www.agilealliance.org/wp-content/uploads/2016/05/IntroductiontotheTechnicalDebtConcep