Verify the shape of terraform output

[niksheridan]

Hi Forum. I have pretty much hit a block on this. I am very much in the process of learning go and terratest 😺

I wanted to test terraform output, as it often the thing I find problematic when passing output to other loops though locals and failing on missing indexes/ attributes and so on.

I have looked at this example which looked pretty straightforward but rather than test for a string or absolute value I was looking to generalise for a type and potentially with a view to move on to perhaps writing functions later to test the value, e.g. regex or integer ranges and so on.

Struct looked highly suitable - arguably perfect for what I am trying to achieve in my first step - so much perhaps, this may have me in a rabbit hole:

type NsgEntry struct {
	access                     string
	destination_address_prefix string
	destination_port_range     []string
	direction                  string
	name                       string
	priority                   int
	protocol                   string
	source_address_prefix      string
	source_port_range          string
}

I was naively hoping the output from terraform (which in my case creates a list of logical objects) could be compared against this struct and verified to have the right 'shape'. For instance in the loop at the bottom here:

func TestTerraformNsgMap(t *testing.T) {
	t.Parallel()

	terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
		TerraformDir: "../examples/terraform-nsg-map-example",
		VarFiles:     []string{"nsg.tfvars"},
		NoColor:      false,
	})
	defer terraform.Destroy(t, terraformOptions)
	terraform.InitAndApply(t, terraformOptions)

	actualTfOutput := terraform.OutputMap(t, terraformOptions, "nsg")

	for _, value := range actualTfOutput {
		assert.IsType(t, value, NsgEntry)
	}

}

The assert is where I run into problems. Also the way that terratest is 'presenting the output' to go is confusing me as it is almost as it if requires unmarshalling from JSON, despite the OutputMap method being used.

I have written more go using the struct directly (based on a book I am reading) and the behaviour of go is very different. In this instance I can access attributes of the object created from the struct like I would do in a python object, which is more familiar to me.

I have searched for similar use cases, but I haven't really found anything suitable which is making wonder if my approach if fundamentally wrong.

Would very much appreciate any guidance on the above if possible.

Thanks in advance.

[gcagle3]

Hello @niksheridan! It seems like your approach to use a struct to represent the expected shape of Terraform output is reasonable. I believe the issue you are encountering is likely due to how Terraform output is returned as type map[string]string (via OutputMap).

In your test function, you're using the IsType function to assert that the value retrieved from the map is of type NsgEntry. This will most likely always fail since the values in the actualTfOutput are of type map[string]string instead of NsgEntry. If you modify the logic so that it performs a type assertion to convert the values of actualTfOutput to NsgEntry structs, then you should be able to perform a proper comparison.

[niksheridan]

Thank you for you reply. Very helpful to affirm my approach seems sensible. I think the value of the tests will in practice transition into the 'conversion step' as I am thinking it will most likely also seem sensible to test type during assignments resulting in the struct comparison becoming somewhat of a formality. I'll post up what I do and I'd be very interested in your feedback if you'd be so kind. Probably wrap this into a function so it doesn't become unbearable.

And yes - I did look more closely at the type the output is received. I am struggling with the static types. Frankly most likely a side effect of me working with python's dynamic typing.

Very helpful indeed, thank you.

[gcagle3]

I'll post up what I do and I'd be very interested in your feedback if you'd be so kind.

I'd be happy to provide feedback once you're ready to share it, absolutely!

[niksheridan]

OK I got it! I will post up a solution later on today when I get time - I did actually use structs and I am basically showing my ignorance of the nuances of go, and probably terratest as well - this post was absolutely critical and I think I have now learned the significance of capitalization, with "and have earned the 'scars' to prove it"!

[niksheridan]

OK I have become somewhat obsessed with finding a solution, and I think I have it here, but it might need some more work:

package tests

import (
	"testing"

	"github.com/gruntwork-io/terratest/modules/terraform"
)

type NsgEntry struct {
	access                     string
	destination_address_prefix string
	destination_port_range     []string
	direction                  string
	name                       string
	priority                   int
	protocol                   string
	source_address_prefix      string
	source_port_range          string
}

func TestNsgStruct(t *testing.T) {
	t.Parallel()

	terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
		TerraformDir: "../terraform/iac1",
		NoColor:      false,
	})

	defer terraform.Destroy(t, terraformOptions)
	terraform.InitAndApply(t, terraformOptions)

	// declare a list of NsgEntry's defined above, into a generic type of map of list of structs
	type NsgOutputs map[string][]NsgEntry

	// the list type becomes a variable of a "map of list of structs"
	var actualTFNsgOutputStruct NsgOutputs

	// this validates the shape of each struct (or object in terraform) on its own and assigns to a pointer
	terraform.OutputStruct(t, terraformOptions, "nsg", &actualTFNsgOutputStruct)

}

I got really confused with the fact there is no assignment for this method, rather the output of is assigned to a pointer (hence the &) which I've never used before. Also in the declaring the struct, when I didn't use a capital in the name of the property it assigns it a null value - so when comparing absolute values in an assert (not shown above of course) it will fail the test. Such a small amount of work took me a great deal of effort but I am trying to learn go - which I am finding quite an unforgiving language - however it is less ambiguous as well.

Here is the sample output:

nsg = {
  "AzureBastion" = [
    {
      "access" = "Allow"
      "destination_address_prefix" = "*"
      "destination_port_range" = [
        "80",
        "443",
      ]
      "direction" = "Outbound"
      "name" = "AzureBastionOut"
      "priority" = 100
      "protocol" = "tcp"
      "source_address_prefix" = "192.168.1.192/26"
      "source_port_range" = [
        "*",
      ]
    },
    {
      "access" = "Allow"
      "destination_address_prefix" = "192.168.1.192/26"
      "destination_port_range" = [
        "80",
        "443",
      ]
      "direction" = "Inbound"
      "name" = "AzureBastionIn"
      "priority" = 110
      "protocol" = "tcp"
      "source_address_prefix" = "*"
      "source_port_range" = [
        "*",
      ]
    },
  ]
  "AzureStorage" = [
    {
      "access" = "Allow"
      "destination_address_prefix" = "192.168.1.128/26"
      "destination_port_range" = "443"
      "direction" = "Inbound"
      "name" = "AzureStorageIn"
      "priority" = 100
      "protocol" = "tcp"
      "source_address_prefix" = "81.174.249.116/32"
      "source_port_range" = [
        "*",
      ]
    },
  ]
  "AzureWidgets" = [
    {
      "access" = "Allow"
      "destination_address_prefix" = "192.168.1.128/26"
      "destination_port_range" = "80"
      "direction" = "Inbound"
      "name" = "AzureWidgetIn"
      "priority" = 100
      "protocol" = "tcp"
      "source_address_prefix" = "81.174.249.116/32"
      "source_port_range" = [
        "*",
      ]
    },
  ]
}

[niksheridan]

Final thing

package tests

import (
	"fmt"
	"reflect"
	"testing"

	"github.com/gruntwork-io/terratest/modules/terraform"
)

type NsgEntry struct {
	Access                     string
	Destination_address_prefix string
	Destination_port_range     []string
	Direction                  string
	Name                       string
	Priority                   int
	Protocol                   string
	Source_address_prefix      string
	Source_port_range          []string
}

func TestNsgStruct(t *testing.T) {
	t.Parallel()

	terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
		TerraformDir: "../terraform/iac1",
		NoColor:      false,
	})

	defer terraform.Destroy(t, terraformOptions)
	terraform.InitAndApply(t, terraformOptions)

	// declare a list of NsgEntry's defined above, into a generic type of map of list of structs
	type NsgOutputs map[string][]NsgEntry

	// the list type becomes a variable of a "map of list of structs"
	var actualTFNsgOutputStruct NsgOutputs

	// this validates the shape of each struct (or object in terraform) on its own and assigns to a pointer
	terraform.OutputStruct(t, terraformOptions, "nsg", &actualTFNsgOutputStruct)

	fmt.Println("ASSIGNMENT INSPECTION:")
	// determine the output of the variable being referenced
	fmt.Println(reflect.TypeOf(actualTFNsgOutputStruct))
	// accessing a variable to demonstrate the shape
	fmt.Println(actualTFNsgOutputStruct["AzureWidgets"][0].Access)

}

This is really good stuff

[niksheridan]

The test highlighted some inadequacies the output my locals blocks were producing - which was exactly what I was originally after. Super pleased - hope this helps someone trying to solve the same problem I had.

nsg = {
  "AzureBastion" = [
    {
      "access" = "Allow"
      "destination_address_prefix" = "*"
      "destination_port_range" = [
        "80",
        "80",
      ]
      "direction" = "Outbound"
      "name" = "AzureBastionOut"
      "priority" = 100
      "protocol" = "tcp"
      "source_address_prefix" = "192.168.1.192/26"
      "source_port_range" = [
        "*",
      ]
    },
    {
      "access" = "Allow"
      "destination_address_prefix" = "192.168.1.192/26"
      "destination_port_range" = [
        "80",
        "80",
      ]
      "direction" = "Inbound"
      "name" = "AzureBastionIn"
      "priority" = 110
      "protocol" = "tcp"
      "source_address_prefix" = "*"
      "source_port_range" = [
        "*",
      ]
    },
  ]
  "AzureStorage" = [
    {
      "access" = "Allow"
      "destination_address_prefix" = "192.168.1.128/26"
      "destination_port_range" = [
        "80",
        "80",
      ]
      "direction" = "Inbound"
      "name" = "AzureStorageIn"
      "priority" = 100
      "protocol" = "tcp"
      "source_address_prefix" = "81.174.249.116/32"
      "source_port_range" = [
        "*",
      ]
    },
  ]
  "AzureWidgets" = [
    {
      "access" = "Allow"
      "destination_address_prefix" = "192.168.1.128/26"
      "destination_port_range" = [
        "80",
      ]
      "direction" = "Inbound"
      "name" = "AzureWidgetIn"
      "priority" = 100
      "protocol" = "tcp"
      "source_address_prefix" = "81.174.249.116/32"
      "source_port_range" = [
        "*",
      ]
    },
  ]
}

[gcagle3]

@niksheridan this is great work! Glad to see that it was able to help you identify some areas for improvement!