What is the recommended folder structure for Terragrunt?

[brikis98]

We got a series of questions from a customer:

1. One repo for infra-live and one for infra-modules, or combine them?
2. One Terraform config for an entire environment, or split out by resource?
3. Whether to use one branch per environment, or a single branch with all environments?
4. How to handle global variables?
5. How to set up the backend (e.g., S3 bucket)?
6. How to connect Terraform configuration to the backend (e.g., S3 state files)?
7. How to handle module defaults?
8. How to handle tagging?

These questions come up often, so I'll post our recommendations below.

---

Tracked in ticket #110425

[brikis98]

One repo for infra-live and one for infra-modules, or combine them?

We recommend separate repos. The primary reason is so you can version your modules and run different versions of those modules in different environments: e.g., you might run v1.0.0 of your eks module in prod while testing out v2.0.0 of the eks module in stage. As described in module versioning:

The easiest way to create a versioned module is to put the code for the module in a separate Git repository and to set the source parameter to that repository’s URL. That means your Terraform code will be spread out across (at least) two repositories:

• modules: This repo defines reusable modules. Think of each module as a “blueprint” that defines a specific part of your infrastructure.
• live: This repo defines the live infrastructure you’re running in each environment (stage, prod, mgmt, etc.). Think of this as the “houses” you built from the “blueprints” in the modules repo.

See here for how to lay out your infra-live repo. And see here for how, in infra-live, to deploy your modules from infra-modules.

One Terraform config for an entire environment, or split out by resource?

Generally speaking, you want to keep Terraform modules small. Bigger modules are slower (e.g., plan can take 10+ minutes), harder to understand (reading 1,000 lines of plan output is hard), less secure (you need broad permissions to make any change), more risky (the blast radius for mistakes is huge), and so on (see large modules considered harmful). Putting all the code for an entire environment in a single module is very likely to result in a large module, so we strongly recommend breaking things down into smaller pieces.

That said, those pieces shouldn't be too small; you wouldn't want to do a single module per resource. You need to find a good balance, grouping things that are typically deployed together, have similar deployment cadences, have similar risk/security profiles, have common team ownership, etc. So, for example, you might have one module that handles all your networking; another module that sets up your data stores (e.g., RDS); another module that handles your orchestration tool (e.g., EKS); and perhaps a bunch of individual modules to deploy apps, each owned by a separate team. See the answer here for more info.

Whether to use one branch per environment, or a single branch with all environments?

We recommend a single branch for all environments, but with versioned modules everywhere, so different environments can deploy different versions. See How to manage multiple environments with Terraform for a comparison of Terraform workspaces, branches, and the Terragrunt approaches (we, of course, recommend the Terragrunt approach).

How to handle global variables?

Typically, you have a hierarchy of variables: some that are truly global across all accounts/environments; some that apply across a single account/environment; some that apply across a single region; some that apply across a set of services; etc.

There are multiple ways to handle this, depending on the use case, but the most common pattern is:

1. See here for how to lay out your folder structure in infra-live to capture this hierarchy.
2. Put reusable variables at the right "level" within that hierarchy in a .hcl file: e.g., account.hcl, region.hcl, networking.hcl, etc.
3. Use read_terragrunt_config and find_in_parent_folders to automatically load the data from the file from the appropriate place in the hierarchy.

For more details and the other options, see Keep your Terragrunt Architecture DRY.

How to set up the backend (e.g., S3 bucket)?

See How should I create the backend (e.g., S3 bucket) for storing Terraform state?.

How to connect Terraform configuration to the backend (e.g., S3 state files)?

See the tutorial here. This will allow you to define your backend configuration in a single place so all your state files are stored in S3 in a folder hierarchy that matches the folder hierarchy of the modules themselves: e.g., the state for prod/us-east-1/networking/vpc/terragrunt.hcl will automatically be stored at prod/us-east-1/networking/vpc/terraform.tfstate in your S3 bucket.

How to handle module defaults?

1. Create Terraform modules in infra-modules that set reasonable default values for all input variables.
2. When deploying those modules in infra-live, you get those defaults automatically, but you can also override them on a per-environment basis if necessary.

How to handle tagging?

See how do I enforce tagging using Terragrunt.

[sandrom]

Regarding modules:
The only addition is that for smaller companies, it is often easier to have those modules in the same repository initially. However, that is an initial state which should gradually move to separate modules. However, I have good experience keeping it in the same repository for quite a while. Eventually, that won't work anymore, and you may have issues developing new features. As things need to be kept compatible anyway, adding feature toggles instead of new versions makes this quite manageable in smaller teams/sizes and can be an alternative approach. That only scales so far, so it depends on what you are looking at.

[gavinying]

I believe gruntwork has given a good general recommendation for folder structures, principles and examples, but of cource developers can further customize the solution to meet their unique needs. Here are some practices we use in our organization,

1. Use local module repo for development - Instead of keep terraform modules in the same terragrunt live repo, we separate them to two (and more) repos as official recommendation, but in order to improve dev efficiency, the terragrunt script can use local-cloned terraform module repo directly during development stage.
2. Enforce tagging for production - All terragrunt scripts must use tagged terraform modules in production env.
3. Segregate terragrunt live repo into different layers - the gist is to segregate the responsibility among teams for whole infrastructure, since Terraform code is not only used by infra team, but project engineers also need to configure their ECS tasks (subject to project needs). In our case, we split terragrunt into infrastructure, data_platform and projects repos (corresponding to infra team, data engineering team and software project team).
4. Segregated terragrunt repos can have different access control for teams and automation tools.

[sandrom]

very good point about segregating into layers. it is also possible in the same repo thanks to codeowners file thankfully and its really important to separate along responsibilities as this enables a lot more self service too :)

[riccardodl]

I also have an use case to submit into the "how do I..." category, for which I would like to see some more documentation about. I find that the examples available do a good job explaining the ideal Terragrunt structure for monolithic applications, where there is only one service that uses a bunch of modules like this.
What is much less clear to me is how to handle multiple microservices that use a collection of modules each.
Something, for example like this (sorry if I use Azure-named services):

environments
│   ├ test 
│          ├ westeurope
│               ├ app1
│                   ├ rg
│                       └─ terragrunt.hcl
│                   ├ web_app
│                       └─ terragrunt.hcl
│                   ├ postgres
│                       └─ terragrunt.hcl
│               ├ app2
│                   ├ rg
│                       └─ terragrunt.hcl
│                   ├ functions
│                       └─ terragrunt.hcl
│                   ├ postgres
│                       └─ terragrunt.hcl

In this case it's quite a bit harder to manage your services, for two reasons.

• If you want to replace a component in you application (for the sake of argument, you want to replace app1/web_app to use Azure Functions instead of App Services) you might also need to modify components imported from other modules (for example the way the app1/postgres module is defined). This is of course expected if you are replacing components in an application, in my experience though, doing it for applications that import many modules and multiplying it for multiple (dev, test, prod) environments, can become quite unwieldy.
• If a module definition changes, for instance an input/output variable changes format (from a flat string to an array, or similar), each .hcl in each component of each application needs to be updated. Versioning modules only "dilutes" the pain, spreading it into multiple PRs rather than one.

One approach I've been trying to use to cope with this is to move the inputs for the individual components into a shared .hcl file a the root of the application.

│          ├ westeurope
│               ├ app1.hcl
│               ├ app2.hcl
│               ├ app1
│                   ├ rg
│                       └─ terragrunt.hcl
│                   ├ web_app
│                       └─ terragrunt.hcl
│                   ├ postgres
│                       └─ terragrunt.hcl
[...]

Where the app1.hcl contains the definitions for all the modules of the application in its input section.

inputs = {
### rg
rg_name = foo
### web_app
container_image = bar:1.0
[...]
}

to then import it into the individual modules of the applications like this:

include "app1_vars" {
  path           = find_in_parent_folders("app1.hcl")
  expose         = true
  merge_strategy = "deep"
}

and leaving the individual modules terragrunt.hcl files "empty". This makes everyday management of an application (like scaling, adding a flat value as input and being able to tell "at a glance" what an application is using) easier.
This approach has its own drawbacks, namely that:

• Dependency variables cannot be defined in the "shared" files, as they would be imported by all the modules of the application, creating circular dependencies (I have to resort to merging variables in the input section when some dependency-dependant variable is needed, with the additional difficulty that deep_merge does not merge array-values.)
• Creating a new application is a very involved process, that requires multiple trial-and-error attempts, as not all the variables can go into app3.hcl, but will have to go into the modules' terragrunt.hcl.

P.S. If someone is wondering why needing to have multiple .hcl files on a two-level hierarchy, instad of a single one containing "everything", it's because terragrunt only supports a single terraform { source = "path/to/tf/module" } statement in a .hcl file.

I would really appreciate if we could get some more documentation on what is the ideal way of handling microservice-oriented architecture with Terragrunt.