What is Infrastructure as Code? And why do you need it?
This article describes what Infrastructure as Code is and what problems Infrastructure as Code can solve. We also discuss the benefits and challenges of Infrastructure as Code and which tools are suitable for Infrastructure as Code.
I am making an assumption: The classic "click" systems engineer is an outdated role. In 10 years at the latest, this role will disappear and Infrastructure as Code will have taken its place. The emerging role will be the Cloud Engineer. So from my perspective, it's worth taking a closer look at Infrastructure as Code.
Let us first discuss the problem scenario and then show you what Infrastructure as Code is and why it is so important.
Table of contents
- The problem scenario
- The problems without infrastructure as code
- What is infrastructure as code?
- What are the benefits of infrastructure as code?
- What are the challenges of infrastructure as code?
- What are infrastructure as code tools?
- Who uses infrastructure as code?
The problem scenario
Developer: Hello System Engineer, in order for our new product feature to work, I need the following change to our web server configuration.
System Engineer: Please describe the change as a step-by-step guide and create a change request.
Developer: OK - I will create a step-by-step guide on how to configure the webserver according to our requirements.
The developer writes step-by-step instructions and creates a change request ticket. Release weekend arrives and the system engineer clicks through the step-by-step instructions. Unfortunately, after working through this step-by-step guide, the application does not work in production. The system engineer calls the developer over the weekend.
System Engineer: Hey Developer, your step-by-step instructions are not working on our production server. Can you help me troubleshoot, I do not know how your application works.
Developer: Of course, I will help you.
Finally, they found a version mismatch of the web server that required another configuration attribute. They were able to fix the problem together on release weekend. But the step-by-step instructions were not updated after that.
The problems
In this scenario there are several problems:
⚡ Manual configuration eliminates traceability of system changes
Change tickets in change management software do not guarantee the traceability of changes in a system. Most likely, they are only used for task management.
⚡ Step-by-step instructions are not sustainable - they become obsolete as soon as they are carried out, and there is no guarantee that they are repeatable
I claim that the step-by-step guide is not updated and runs alongside the current configuration.
⚡ The classical demarcation between software development and system engineering does not automatically lead to a technical quality gate.
The strict demarcation of systems engineering and development is, in my opinion, just an organizational pseudo-quality gate. I assume that in most cases the developer knows the required system configuration better than the system engineer.
⚡ Organizational separation slows down the time to market of your product development.
This organizational separation and the heavyweight change management process slows down time to market for new features or bug fixes because there is at least one handoff from development to operations.
To address these problems, Infrastructure as Code comes into play.
What is Infrastructure as Code?
With Infrastructure as Code, system changes are managed and implemented through code and automation, rather than through manual change processes. So Infrastructure as Code means that every single change to the infrastructure is described and executed with code.
The code for the infrastructure is written using the same tools that a developer uses:
- A IDE or editor (like VS Code) with an appropriate programming and language like TypeScript, Go, etc.
- A source code management software like Git with a suitable flow that supports a code quality gate.
- A CI/ CD toolstack (like GitLab Runner, GitHub Actions, Bitbucket Pipelines, Jenkins) that tests and executes the infrastructure code.
Infrastructure as Code example
The following example code snippet shows you an example of how to deploy a Linux web server in an Azure virtual machine and assign it a public IP address. The underlying Infrastructure as Code tool is Pulumi (see details below What are good Infrastructure as Code tools?).
The complete project can be found under the following link:
pulumiWhat are the benefits of Infrastructure as Code?
When you manage your infrastructure changes with code, you gain the following benefits:
- Traceability of every system change through the version management tool (e.g. git).
- Repeatable automation of provisioning and changing system resources.
- The same tooling for managing and deploying infrastructure changes and software development.
- Automated quality checks through the same routines of the CI/CD pipeline, such as linting, unit tests, property tests, or integration tests.
- Shorter time to market as there is no organizational gap between "Dev" and "Ops" and no handover.
- Portability of your system configuration, as any configuration can be repeated on different target system.
What are the challenges of Infrastructure as Code?
Infrastructure as Code needs a rethink in an organizational and a technical dimension.
🔸 Infrastructure as Code is part of DevOps; it forces organizational change
In my opinion, Infrastructure as Code forces us to rethink the old, traditional IT organizational structure. Like application code, infrastructure code is also driven by the functional and non-functional requirements of the corresponding product. So writing infrastructure code should also be the responsibility of the product team. This is where the cross-functional team approach comes into play. Ideally, there should be no organizational gap between the product team and the systems engineering team that writes the infrastructure code for the product team. The product team has the ability and responsibility to write and release the infrastructure code.
🔸 A combination of strong Software and system engineering skills are needed
Infrastructure as code requires strong software engineering while having deep systems engineering skills. This means that you should have a deep understanding of the underlying infrastructure provider and be able to write and test your infrastructure code in a real programming language. So, there must be a system engineer with software engineering knowledge to write this infrastructure code.
🔸 A cloud strategy is required
If your infrastructure runs on one of the big three cloud providers, you are ready for Infrastructure as Code. Otherwise, it depends. There is broad support for Infrastructure as Code among the following infrastructure providers and platforms:
- Amazon Web Services
- Microsoft Azure
- Google Cloud Platform
- IBM Cloud
- DigitalOcean
- Oracle Cloud Infrastructure
- VMware vSphere
- OpenStack
What are good Infrastructure as Code tools?
There are a lot of good Infrastructure as Code tools on the tool market.
Pulumi
Pulumi is an Infrastructure as Code tool that allows you to write infrastructure code in a "standard" programming language such as JavaScript / TypeScript (Node.js), Python, Go or C# / F# (. NET). This allows for more flexibility and less complexity than domain-specific languages.
Support for "standard" programming languages promotes a broad development ecosystem and provides for modern application development that allows for sharing and reuse of best practices.
In addition, it is possible to write tests for infrastructure code to validate infrastructure changes before they are deployed to production.
Last but not least, Pulumi provides a cloud service to enforce policy-as-code, auditing, and access controls, resulting in fine-grained team-wide visibility.
Get a sneak peek at Pulumi with the following video
HashiCorp Terraform
HashiCorp Terraform is a mature Infrastructure as Code platform. It has its own HashiCorp Configuration Language (HCL) to describe the configuration of the infrastructure in a declarative way. In addition to writing declarative configuration files, Terraform CLI allows you to plan and predict changes and provides you with a dependency graph. Terraform supports configuration state management to keep track of metadata and improve performance for large infrastructures.
There is a huge Terraform Registry with 1000+ for your cloud platform and services for your configuration to provision infrastructure.
To get started with Terraform, check out the following video from IBM technology 👇
RedHat Ansible
Ansible is an open-source product and one of the most widely used configuration management tools. It automates cloud provisioning, configuration management, and application deployment.
Ansible executes configuration tasks defined in YAML files or playbooks. The configuration tasks are idempotent, meaning they can be applied multiple times without changing the result beyond the initial state. This is what makes Ansible so reliable and sets it apart from some other configuration management tools.
To get started with Ansible, watch the following video from IBM technology 👇
Chef Infra
Chef Infra is a powerful, policy-based configuration management software that eliminates manual effort and ensures that infrastructure remains consistent and compliant throughout its lifecycle-even in the most complex, heterogeneous, and large-scale environments.
With Chef Infra, you can write configurations and policies as code that is testable, enforceable, and can be delivered at scale as part of automated pipelines.
Chef Infra ensures that configurations are changed only when a system deviates from the desired defined state, and automatically corrects configuration deviations as needed.
With Chef Infra, you can manage Windows and Linux systems running on-premises, ARM systems running in the cloud, or Macs laptops running at the edge of the same way.
To learn about Chef Infra, watch the following video 👇
Who uses infrastructure as code?
If we trust the testimonials, case studies, or success stories of the Infrastructure as Code tools described above, the following companies use Infrastructure as Code in their daily product development:
Pulumi Customers
- Atlassian
- Lemonade
- Lykke
- Mercedes Benz Development and Research
- SANS Institute
- WhyLabs
- Snowflake
- Sourcegraph
Comments ()