Start small using OPA

Start small using OPA

As I mentioned in my previous blog post, my first OPA policy was just to catch one simple parameter, if we have in the S3 Terrafom module set the force_destroy = true.

Here’s a simple OPA policy that will catch this dangerous configuration:

package terraform.plan

deny[msg] {
    # Find all resources in the plan
    resource := input.resource_changes[_]
    
    # Check if it's an S3 bucket
    resource.type == "aws_s3_bucket"
    
    # Look for force_destroy in the configuration
    resource.change.after.force_destroy == true
    
    msg := sprintf("S3 bucket '%s' has force_destroy set to true. This is dangerous as it allows bucket deletion even when not empty.", [resource.address])
}

This policy works by:

  1. Looking through all resource changes in the Terraform plan
  2. Finding any resources of type aws_s3_bucket
  3. Checking if force_destroy is set to true in the configuration
  4. If found, it generates a denial message with the bucket’s address

When this policy is evaluated against a Terraform plan, it will fail if any S3 bucket has force_destroy = true, preventing the potentially dangerous configuration from being applied.

Testing the Policy

Great, but how we can test this policy? We have two options: using the OPA CLI directly or using conftest, a tool that makes it easy to test configuration files using OPA policies.

Using OPA CLI

You can test the policy directly using the OPA CLI. First, install OPA:

# For macOS
brew install opa

Then, generate a Terraform plan in JSON format:

terraform plan -out=tfplan
terraform show -json tfplan > plan.json

Now you can evaluate the policy against the plan:

opa eval --format pretty --data policies/terraform/force_destroy.rego --input plan.json "data.terraform.plan"

The output will show you any violations of the policy. While this works, it requires more manual setup and doesn’t provide the nice formatting and additional features that conftest offers.

Using Conftest

Conftest is a wrapper around OPA that makes it easier to test configuration files. Here’s how to get started:

  1. First, install conftest. You have two options:

    a. Using Homebrew on macOS:

    brew install conftest
    

    b. Or download the binary directly:

    # For macOS
    LATEST_VERSION=$(wget -O - "https://api.github.com/repos/open-policy-agent/conftest/releases/latest" | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/' | cut -c 2-)
    ARCH=$(arch)
    SYSTEM=$(uname)
    wget "https://github.com/open-policy-agent/conftest/releases/download/v${LATEST_VERSION}/conftest_${LATEST_VERSION}_${SYSTEM}_${ARCH}.tar.gz"
    tar xzf conftest_${LATEST_VERSION}_${SYSTEM}_${ARCH}.tar.gz
    sudo mv conftest /usr/local/bin
    
  2. Generate a Terraform plan in JSON format:

terraform plan -out=tfplan
terraform show -json tfplan > plan.json
  1. Now you can test the plan against your policy:
conftest test plan.json -p policies/terraform

If any S3 bucket in your plan has force_destroy = true, conftest will show a failure with the message we defined in the policy. For example:

FAIL - plan.json - terraform.plan - S3 bucket 'module.s3.aws_s3_bucket.example' has force_destroy set to true. This is dangerous as it allows bucket deletion even when not empty.

You can customize the output format using the --output flag. Conftest supports several output formats:

# Standard output (default)
conftest test plan.json -p policies/terraform

# JSON output
conftest test plan.json -p policies/terraform --output json

# TAP (Test Anything Protocol) output
conftest test plan.json -p policies/terraform --output tap

# Table output
conftest test plan.json -p policies/terraform --output table

This makes it easy to catch dangerous configurations before they’re applied to your infrastructure. You can integrate this into your CI/CD pipeline to automatically check all Terraform plans before they’re applied.

Integration with CI/CD

One of the most powerful aspects of using OPA + conftest is its seamless integration with various CI/CD platforms. Here are some common integration points:

Atlantis Integration

Atlantis is a popular tool for automating Terraform workflows through pull requests. You can integrate conftest with Atlantis by adding a pre-plan hook that runs your OPA policies. This ensures that all Terraform plans are validated against your policies before they’re even proposed for review.

Here’s a simple example of how to configure Atlantis to run conftest:

# atlantis.yaml
version: 3
projects:
- name: my-terraform-project
  dir: .
  workflow: custom
workflows:
  custom:
    plan:
      steps:
      - run: terraform plan -out=tfplan
      - run: terraform show -json tfplan > plan.json
      - run: conftest test plan.json -p policies/terraform

Other CI/CD Platforms

The same approach can be applied to other CI/CD platforms:

  • GitHub Actions: Add a step in your workflow to run conftest after generating the Terraform plan
  • GitLab CI: Include conftest in your pipeline stages to validate Terraform configurations
  • Jenkins: Add conftest as a build step in your pipeline
  • Azure DevOps: Integrate conftest into your YAML pipelines

Following the best practice of separating terraform plan and terraform apply into different pipeline stages, conftest checks should be placed between them. This creates a safety net that catches policy violations before any infrastructure changes are made.

Conclusion

Working with Terraform plans can be challenging, especially when dealing with complex infrastructure setups. Whether you’re using Terragrunt, managing a monorepo, or working with nested modules, the plan output can quickly become overwhelming with hundreds or thousands of lines of changes. It’s easy to miss critical configurations like force_destroy = true buried in the noise.

This is where Policy as Code with OPA and conftest shines. Instead of manually reviewing every line of the plan output, you can write specific policies that focus on what matters most to your organization. The policy we created today is just a simple example, but it demonstrates how you can:

  • Automatically catch dangerous configurations
  • Reduce the cognitive load of reviewing complex plans
  • Enforce consistent standards across your infrastructure
  • Make the review process more reliable and less error-prone

As your infrastructure grows, these benefits become even more valuable. You can gradually add more policies to cover other important aspects of your infrastructure, such as security settings, cost controls, or compliance requirements. Each policy acts as a guardrail, helping you maintain control over your infrastructure even as it becomes more complex.

Remember, the goal isn’t to replace human judgment but to augment it. By automating the detection of known issues, you free up time and mental energy to focus on the unique aspects of your infrastructure that require human attention.

Next Steps

This is just the beginning of our journey with OPA and Terraform. In the upcoming posts, we’ll explore more advanced use cases:

  1. Enforcing Tags: Learn how to write OPA policies that ensure all resources have the required tags. This is crucial for cost allocation, security, and compliance. We’ll cover how to enforce both the presence of specific tag keys and their expected values.

  2. Cloud Cost Management with OPA: Discover how to implement the shift-left paradigm for cloud costs using OPA and Terracost. We’ll explore how to:

    • Estimate infrastructure costs before deployment
    • Set cost thresholds and alerts
    • Enforce cost-related policies in your CI/CD pipeline
    • Prevent expensive misconfigurations early in the development process

Stay tuned for these upcoming posts as we dive deeper into infrastructure policy enforcement and cost management.