Skip to main content

Terraform State

Terraform stores state about your managed infrastructure and configuration. This state is used by Terraform to map real world resources to your configuration, keep track of metadata, and to improve performance for large infrastructures. For a detailed understanding of terraform state, please visit the official docs.

Where is the state?

By default the terraform state for all Configurations is stored in Kubernetes secrets located in the controller namespace. The following template is used as the backend.

  • Namespace is always the controller namespace.
  • Suffix is the Configuration UUID.
  • Note the kubernetes backend adds a prefix tfstate- so the state secrets will be named tfstate-UUID.
var backendTF = `
terraform {
backend "kubernetes" {
in_cluster_config = true
namespace = "{{ .controller.namespace }}"
secret_suffix = "{{ .controller.suffix }}"
}
}

How to change state backend?

tip

Note the ability to override the backend is only available with version >= v0.3.1

While using Kubernetes as a backend has it's benefits in terms of ease of use, there's a few downsides as well.

  • You need to ensure backups are performed on the state secrets.
  • Its harder operate or manipulate the terraform state, using taints for example.
  • The terraform state is not versioned so rollbacks are harder to performed.
  • You are unable to reference the state using remote_state_data resource.
  • The terraform state is ultimately bound to the Cluster.

Platform administrators can change the backend using the following steps.

Create a template for the backend to use

  1. Create a kubernetes secret in the controller namespace containing the template
cat <<EOF > backend.tf
terraform {
backend "s3" {
bucket = "terranetes-controller-state"
key = "cluster_one/{{ .namespace }}/{{ .name }}
region = "eu-west-2"
access_key = "AWS_ACCESS_KEY_ID"
secret_key = "AWS_SECRET_ACCESS_KEY"
}
}
EOF
tip

We inject the entire Configuration resource into the context on the template, so you can reference anything side via configuration.PATH

Create a kubernetes secret from the above file

kubectl -n terraform-system create secret generic backend-s3 --from-file=backend.tf=backend.tf
  1. Update the controller to use the backend template

If you are using the helm chart you simply have to update

controller:
# Name of a secret in the controller namespace which contains the template to use
# for the backend state
backendTemplate: backend-s3

If you are deploying the controller yourself, update the --backend-template=backend-s3 command line flag.