Using iacbot for Operational Control of IaC

Rob Schoening

Using Automation to Correct Configuration Mistakes Before They Get to Production

A few months ago, we introduced iacbot to help developers easily find and fix misconfigurations in their infrastructure as code (IaC) – including Terraform and CloudFormation. Watch our latest demo and read below to learn how to use iacbot for automated testing and policy management, giving you operational control to scale your use of IaC safely.

While deploying untested and unvalidated application code is unthinkable in 2021, many teams will happily deploy IaC in their CI/CD pipelines without any structured test automation.

Configuration mistakes are easy to make when writing IaC. Security policies, compliance policies, best practices and company policies are difficult to detect and enforce without automated tests.

What is the impact?

The result is predictable. As soon as functional requirements are met and the cloud resources do what teams need them to do, effort stops because they receive no immediate feedback on their changes.

Organizations are wholly dependent on downstream cloud configuration posture management (CSPM) tools to detect misconfigurations and violations. Those tools are typically run by cloud security teams who are quite a distance from the teams authoring the IaC and managing cloud resources. If the development teams don’t receive fast feedback on their changes, the result is a security backlog that grows quickly.

More problematic still, many configuration mistakes are difficult to fix once they are in place. Immutable infrastructure like stateless compute services are easy, but stateful services like databases, storage, network configuration can be extremely expensive to reconfigure after the fact. Even the simplest of settings, such as TLS or database encryption at rest, can take weeks or months to fix if they are allowed to be deployed before the problems are detected and fixed.

The solution to this is obvious: add automated tests and policy control to CI/CD so that incorrect or out-of-policy IaC can’t be merged or deployed. This solution isn’t controversial, yet for most, nothing is being done.

We hope for the best and curse the result.

So what is our approach and how does our product work?

iacbot connects directly to GitHub and GitLab through their respective marketplaces. There is nothing to install unless you want to run it in your environment.

You can get started by going to https://get.soluble.cloud and clicking on “Get Started” in the upper right.

It takes just a minute to choose the repositories you want to connect to iacbot.

iacbot will then begin performing initial scans to gather insights of your security posture of your Terraform, CloudFormation and Kubernetes manifests. Within a minute or two you will have a dashboard of prioritized findings.

iacbot dashboard

These stats will be updated daily at a minimum and on demand as code changes are pushed to your source control system.

If you select a specific repo, you’ll be taken to a repository-specific view that aligns with the team making the changes.

iacbot repo view

Findings have clear summaries of the problem, guidelines for how to resolve the issues, and links that take you directly to the line of source code that generated the finding so that it can be fixed quickly.

iacbot code change

But most importantly, the findings are delivered directly to pull requests (PRs) where they cannot be missed by the IaC authors.

Developers want to see IaC violations just as they would expect to see CI test results.

No development team wants to switch context from their normal workflow and use a separate tool for security testing.

The following shows a simple GitHub PR. The IaC developer changed the instance type of the RDS database along with a change that made the instance public.

iacbot pr

iacbot detected these changes and failed the PR status check so that the change can’t be merged to the main branch.

iacbot also added a PR comment that clearly spells out the reason for the failure, along with links to the guidelines to resolve the problem.

Without iacbot in the PR flow, this change would likely have been merged and applied to the deployed environment.

Even if there are platform experts in the code review process, it is very easy for configuration mistakes to go undetected. Reviewing Terraform and CloudFormation is a tedious process that is difficult for people to perform routinely. It is not at all like reviewing application code. The settings are arcane and it is difficult to reason about the impact of changes.

The result? Too often, “looks good to me,” when in reality the change is not good at all.

If you do not have iacbot and if you do not have a cloud security team that is operating a cloud security posture management (CSPM) tool to catch configuration errors in the cloud provider’s control plane, the problem would not be detected at all.

Even if your cloud security team has a well-maintained CSPM system in place, those teams are finding that alerts for violations aren’t enough. It is labor intensive and slow to assign CSPM violations back to the teams that caused them, and the violations lack the context that the developers need to fix the problem.

Developers want fast feedback on their changes. The longer the interval between the proposed change and the detection, the greater the cost to fix.

If the production database has started to collect data, there is often no easy fix that avoids downtime and labor cost. There is also no guarantee that the people who made the change are still available if too much time elapses. They may have moved on to other activities.

The cost of a trivial one-line fix in development costs almost nothing to fix, but can cost 10x or more if it is not caught early.

Everything we are discussing here is sacrosanct in modern development. The results are in, and test automation in CI/CD is the single most efficient way to increase security, quality and efficiency of software services.

Now with iacbot, and just a few minutes of time, you can realize these benefits for your cloud infrastructure as well.

Try it out, and let us know what you think.