Skip to content

Terraform State

Context

Terraform generates State, which is basically a magical JSON file you don't want to have to edit manually. These files are then stored in locations like GCS, S3 or even as Kubernetes secrets if you so desire.

Technology used

Google Cloud Terraform GitHub Actions

Project Breakdown and solution

We needed a better way to store our state. As part of creating a project, we would provision a bucket called <project-name>-terraform-state which meant that each and every project would have a bucket per environment. This made managing IAM and data blocks a nightmare.

The solution was to deploy a single project which would hold all the state files, develop a module to create these buckets with a lifecycle rule to keep 100 non-current versions and then delete these old versions after 100 days.

These buckets are then deployed via GitHub actions and a taskfile

What I did

  • Identified the issue
  • Migrated all 200 state buckets from across projects in to grouped buckets
  • Re-wrote the terraform to point to these new buckets (this was painful)
  • Set up a terraform module to make these buckets the way we wanted to
  • Designed the taskfile
  • Setup CI and IAM for creating buckets