I ran into the problem that I wanted to make an existing root module (I'll call this source) a child module of another root module (I'll call this destination). Both modules used a remote state backend. In my case a GCP storage bucket.

Example

Let's say we have 2 root modules. redis & databases. The databases module contains a postgresql child module. We want to make the redis module a child module of the databases module.

This requires that we move the state of the existing source module into the destination module's state.

Our before & after Terraform module state:

(SVG version for less pixel artifacts)

Before

File structure

~/
├── databases/
│   └── main.tf
├── redis/
│   └── main.tf
└── modules/
    └── postgresql/
        └── main.tf

Code

~/databases/main.tf:

...

resource "module" "postgresql" {
  source = "../modules/postresql"
}

~/redis/main.tf:

...

resource "google_redis_instance" "my_redis_instance" {
  ...
}

resource "random_password" "my_redis_pw" {
...
}

~/modules/postgresql/main.tf:

...

resource "google_sql_database_instance" "postgresql" {
  ...
}

resource "random_password" "my_sql_pw" {
...
}

After

File structure

~/
├── databases/
│   └── main.tf
└── modules/
    ├── redis/
    │   └── main.tf
    └── postgresql/
        └── main.tf

Code

~/databases/main.tf:

...
// Has the resource address "module.redis" 
resource "module" "redis" {
  source = "../modules/redis"
}

resource "module" "postgresql" {
  source = "../modules/postresql"
}

~/modules/redis/main.tf:

...

resource "google_redis_instance" "my_redis_instance" {
  ...
}

resource "random_password" "my_redis_pw" {
...
}

~/modules/postgresql/main.tf:

...

resource "google_sql_database_instance" "postgresql" {
  ...
}

resource "random_password" "my_sql_pw" {
...
}

Making module child of other module

Resolving the problem described above can be achieved by running a combination of commands:

• terraform state pull: Pull the remote states of the source & destination modules down as state files to your local drive.
• terraform state mv: Move all resources from the pulled down local source module state to the local destination module state.
• terraform state push: Push the local destination module state file back up to the remote.

As explained in this Stackoverflow answer.

This approach works fine for small modules that don't contain a whole lot of resources. It can be cumbersome if that's not the case for you. When you're working with large modules. Especially if you need to move multiple such modules.

I created a small script that automates this process.

It will also create a backup of your existing state before changing anything. Just in case (Terraform additionally also creates backup files before every terraform state mv).

It takes in a couple of arguments:

1. The directory of our destination module. In our example: ~/databases.
2. The directory of our source module. In our example: ~/redis.
3. The resource address inside the destination module where the source module should be moved to. In our example: module.redis.

And the optional option -dry-run. Which does what it indicates. Does a dry run. No state is moved, but you can see what the script will do on an actual run.

migrate-state.sh:

#!/usr/bin/env bash
set -euo pipefail

# Support aborting via SIGINT, without this bash will not exit the for loop until it's finished
trap 'exit 0' INT

# Usage example:
# bash migrate-state.sh '/home/myuser/terraform/destination-module' '/home/myuser/terraform/source-module' 'module.source' 'config/dev_backend.tfvars' 'config/dev_backend.tfvars'
# Arguments:
# $1 - destinationModuleDirectory: The directory in which the module resides where state should be moved to
# $2 - sourceModuleDirectory: The directory in which the module resides where state should be moved from
# $3 - destinationModuleSourceParentAddress: The resource address inside the destination module that is used as parent for all resources that are moved from the source module
# $4 (optional) - sourceModuleBackendConfig: The value that should be used for the "-backend-config" parameter when running  terraform init for the source module
# $5 (optional) - destinationModuleBackendConfig: The value that should be used for the "-backend-config" parameter when running  terraform init for the destination module
# Options:
# -dry-run
#   Show changes that will be made, does not actually change anything

function migrateState() {
  local destinationModuleDirectory="${1:?'The destinationModuleDirectory argument is missing'}"
  local sourceModuleDirectory="${2:?'The sourceModuleDirectory argument is missing'}"
  local destinationModuleSourceParentAddress="${3:?'The destinationModuleSourceParentAddress argument is missing'}"
  local sourceModuleBackendConfig="${4:-''}"
  local destinationModuleBackendConfig="${5:-''}"

  local isDryRun=0
  for arg in "${@}"
  do
    if [[ "${arg}" == '-dry-run' ]];
    then
      isDryRun=1
      break
    fi
  done

  local lightBlue='\e[38;5;26m'
  local colorEnd='\e[0m'

  local destinationModuleLocalStateFile="${destinationModuleDirectory}/destination-module.tfstate"

  cd "${destinationModuleDirectory}" || false

  local destinationTerraformInitArgs=''
  if [[ "${destinationModuleBackendConfig}" != '' ]];
  then
    destinationTerraformInitArgs="-backend-config=${destinationModuleBackendConfig}"
  fi
  terraform init "${destinationTerraformInitArgs}"

  terraform state pull > "${destinationModuleLocalStateFile}"
  cp "${destinationModuleLocalStateFile}" "${destinationModuleLocalStateFile}.bak"

  cd "${sourceModuleDirectory}" || false
  local sourceTerraformInitArgs=''
  if [[ "${sourceModuleBackendConfig}" != '' ]];
  then
    sourceTerraformInitArgs="-backend-config=${sourceModuleBackendConfig}"
  fi
  terraform init "${sourceTerraformInitArgs}"
  terraform state pull > "${sourceModuleDirectory}/source-module.tfstate.bak"

  for sourceResourceAddress in $(terraform state list)
  do
    local destinationResourceAddress="${destinationModuleSourceParentAddress}.${sourceResourceAddress}"
    printf "Moving ${lightBlue}%-100s${colorEnd} to ${lightBlue}%-120s${colorEnd}\n" "${sourceResourceAddress}" "${destinationResourceAddress}"

    if [[ ${isDryRun} == 1 ]];
    then
      terraform state mv -state-out="${destinationModuleLocalStateFile}" -dry-run "${sourceResourceAddress}" "${destinationResourceAddress}"
    else
      terraform state mv -state-out="${destinationModuleLocalStateFile}" "${sourceResourceAddress}" "${destinationResourceAddress}"
    fi

    printf '\n'
  done

  if [[ ${isDryRun} == 0 ]];
  then
    cd "${destinationModuleDirectory}" || false

    terraform init "${destinationTerraformInitArgs}"
    terraform state push "${destinationModuleLocalStateFile}"
  fi
}

migrateState "${@}"

The script is also available on GitHub.

To get back to our example. We'd run

bash migrate-state.sh '~/databases' '~/redis' 'module.redis' -dry-run

to see what will happen.

And then remove the -dry-run option:

bash migrate-state.sh '~/databases' '~/redis' 'module.redis'