Infrastructure as Code (IaC) - there are not only advantages but also new challenges - part one.

Infrastructure as Code (IaC for short) is currently the leading way of implementing solutions in cloud environments. At Hosters, we treat it as a deployment standard, leaving manual, so-called 'clickable' cloud service configurations for creating very small, usually short and non-developmental projects or, more often, for experimental projects, so-called PoCs (Proof of Concept).

By design, IaC is intended to help solve key cloud resource management problems, and the list of these problems (or challenges, as one prefers) is long. We will list just a few of them below - in our opinion, the most significant ones, revealing themselves very quickly when least expected as a consequence of creating cloud resource configurations manually, e.g., using wizards in the web console.

So what could put us at risk?

• Loss of control over the configuration of services and resources—this problem becomes particularly acute when managing a huge-scale infrastructure, e.g., tens of thousands of resources.
• A time-consuming and costly manual process of changing or updating the configuration of often hundreds or thousands of different cloud components, with the added risk of human error.
• Discrepancies and inconsistencies between configurations of instances of the same services or cloud resources.
• Lack of an easy way to restore previous settings after making erroneous changes.
• Lack of ability to track the history of changes to cloud component configurations.
• Before making them, there is a lack of ability to check and review modifications, anticipated post-deployment results, and potential side effects on the cloud environment.
• Lack of documentation of the infrastructure itself or the changes made.
• Lack of a consistent and reliable mechanism for updating cloud infrastructure configurations.
• Lack of control over dependencies between cloud resources.

The answer to these problems is a method known as Infrastructure as Code (IaC), implemented in various forms: services such as AWS CloudFormation, tools based on dedicated Domain Specific Language (DSL) languages such as Terraform, or derived tools such as AWS CDK (Cloud Development Kit) based on imperative general-purpose programming languages such as Python or Java.

However, the proliferation of IaC and the application of this relatively young technology raises new challenges. The lack of developed, mature and repeatedly proven design patterns, which almost every technology suffers in its age of 'maturity', causes many of our customers to rely on knowledge gleaned from unproven sources or experiment by learning from their mistakes. Meanwhile, this is precisely the spectrum of knowledge in which Hosters already has expert experience and can help avoid potential problems and thus significantly reduce costs and speed up cloud project implementations based on years of experience gathered from implementing and maintaining hundreds of cloud projects using IaC.

When it comes to standalone IaC implementations, our customers often struggle with the following pain points:

• "Monolithic code hell" ("Monolithic Hell") - the case where all cloud services and resources are deployed and defined in one very large piece of code (monolith), with no separation into independent parts, layers of related components or groups of services and resources with clearly defined roles, areas or responsibilities. This problem manifests itself particularly painfully in various errors during deployments on production environments in the AWS CDK (Cloud Development Kit), often related to exceeding the limits of individual services that are part of the so-called resource stack managed by this code monolith.
• A derivative of the above problem is the high risk of confusion when implementing changes to the resource stack created from the code monolith. A minor or unintentional typo in the CDK code can cause a chain reaction, a cascade of uncontrolled changes, and even the accidental destruction of critical infrastructure elements that make up the stack.
• The overly complex hierarchy of IaC code and the use of advanced, but often inadequate, mechanisms of a particular language or tool. The use of multi-level inheritance or polymorphism of object-oriented languages in the case of application code implementation usually works great; however, in the case of infrastructure code implementation, these will not necessarily be the right options and choices. We have more than once encountered cases where, after the author(s) of the infrastructure code left, no one from the client's team dared to touch the code left behind for fear of the consequences of making changes whose side effects on the production environment were not easy to predict and risked accidental destruction or modification of application-critical services. The level of complexity and the range of dependencies between IaC projects meant that understanding how even a seemingly simple change would work was not entirely trivial and required navigating and switching between several projects to get to the bottom. This is when companies turn to Hosters for support, and usually, after analyzing the code and the actual configuration of the services on the cloud, our DevOps safely make the required modifications.
• Lack of naming conventions and consistency in the structure of an IaC project - the lack of accepted standards when it comes to organizing the structure and naming of packages, modules, and classes, or the lack of consistency in applying and enforcing these standards makes managing such a project quite a challenge over time. Then, we often encounter cases of repeatedly copying the same or very similar pieces of code. It is not uncommon for a new infrastructure developer to have trouble finding and using a component that has already been developed - so he re-invents the wheel by implementing the same functionality again.
• Discrepancies between the source of truth ('single source of truth'), which should be the current version of the IaC code in the repository, and the actual implementation on cloud environments, especially production environments. Most often, this is the unintended result of one or two undesirable situations:
  A uniform mechanism for deploying IaC code and applying code changes from infrastructure developers' local computers is lacking.
  The ability to make manual/manual configuration modifications - usually from a web console.
• Lack of knowledge and experience with a particular programming language or IaC tool - where the result is usually the implementation of code in a way that is not always fully optimal due to the use of inappropriate programming language constructs, e.g., repeated copying of the same code fragments or creation of own components instead of using already existing constructs built into the language or IaC technology.

So, how do you deal with the above problems? We will discuss this in the second part of the post soon.

Author: Adam Smolnik, Senior Cloud Architect at Hostersi and AWS Community Hero.