"eachKeyLike" verification changing behaviour between PACT Provider versions

[DanJKing]

We seem to have had a scenario where updating our provider side library is causing a failing in verification on an eachKeyLike matcher.

The endpoint

Example response JSON:

{
"abcde": {
      "name": "productSet",
      "value": "my-set-id"
    }
}

...where there will be any number of keys like abcde , but with different values based on the body sent to the endpoint. The structure of the object hanging off the key will always be consistent, with a name and a value.

i.e. this would also be a valid response

{
"abcde": {
      "name": "somename",
      "value": "somevalue"
    },
"rfgd": {
      "name": "somename2",
      "value": "somevalue2"
    }
}

Consumer Matching Code:

On the consumer side, the team have been using the eachKeyLike matcher to account for these behaviours:

val providerResponseBody = PactDslJsonBody()
            .eachKeyLike("abcde")
            .stringType("name", "productSet")
            .stringType("value", setId)
            .closeObject()

With version "au.com.dius:pact-jvm-consumer-junit:4.0.10".

On the provider side, when using "au.com.dius.pact.provider:junit5:4.1.42" we see the contract test passing.

However, when attempting to migrate the provider side to "au.com.dius.pact.provider:junit5:4.6.0" we see the following failure:

1) Verifying a pact between digital-merch.product-merch and search.locations - POST /api/locations/pairs with set ids has a matching body

    1.1) body: $ Actual map is missing the following keys: abcde

        {
        -  "abcde": {
        +  "pj41": {
            "name": "productSet",
            "value": "my-set-id"
          }
        }

So, from my reading, it's now complaining that in the response it got back from the endpoint during the verification, it received the pj41 key, not the abcde key.

Are you able to explain why this is happening? It looks as though the later provider side version is doing some kind of value matching in the verification now, whereas clearly in the previous version we used it did not.

The PACT is generated from the consumer side to the designated Spec, and that code has not changed, so our assumption was updating the Provider side library should not cause this kind of issue?

Thanks

[rholshausen]

Can you provide your Pact file and debug logs?

[DanJKing]

Hi - see below:

Debug logs from the verification step in our pipeline:

POST /api/locations/pairs with set ids
        returns a response which
          has status code 200 (OK)
          has a matching body (FAILED)
    Failures:
    1) Verifying a pact between digital-merch.product-merch and search.locations - POST /api/locations/pairs with set ids has a matching body
        1.1) body: $ Actual map is missing the following keys: abcde
            {
            -  "abcde": {
            +  "pj41": {
                "name": "productSet",
                "value": "my-set-id"
              }
            }
Gradle Test Executor 1 finished executing tests.
> Task :test FAILED
LocationsContractTest > pactVerificationTestTemplate$contract_tests_test(PactVerificationContext) > Pact between digital-merch.product-merch (7ed80124) and search.locations - POST /api/locations/pairs with set ids FAILED
    java.lang.AssertionError: Pact between digital-merch.product-merch (7ed80124) and search.locations - Upon POST /api/locations/pairs with set ids 
    Failures:
    1) Verifying a pact between digital-merch.product-merch and search.locations - POST /api/locations/pairs with set ids has a matching body
        1.1) body: $ Actual map is missing the following keys: abcde
            {
            -  "abcde": {
            +  "pj41": {
                "name": "productSet",
                "value": "my-set-id"
              }
            }
        at au.com.dius.pact.provider.junit5.PactVerificationContext.verifyInteraction(PactVerificationContext.kt:75)
        at LocationsContractTest.pactVerificationTestTemplate$contract_tests_test(LocationsProviderContractTest.kt:24)
1 test completed, 1 failed

And the PACT uploaded to the broker:

{
  "provider": {
    "name": "search.locations"
  },
  "consumer": {
    "name": "digital-merch.product-merch"
  },
  "interactions": [{
    "_id": "c343ef7729664e473a8b425954fe94cf4e55afc8",
    "description": "POST /api/locations/pairs with set ids",
    "request": {
      "method": "POST",
      "path": "/api/locations/pairs",
      "headers": {
        "Content-Type": "application/json"
      },
      "body": [{
        "name": "productSet",
        "value": "my-set-id"
      }],
      "matchingRules": {
        "body": {
          "$[0].name": {
            "matchers": [{
              "match": "type"
            }],
            "combine": "AND"
          },
          "$[0].value": {
            "matchers": [{
              "match": "type"
            }],
            "combine": "AND"
          }
        }
      }
    },
    "response": {
      "status": 200,
      "headers": {
        "Content-Type": "application/json"
      },
      "body": {
        "abcde": {
          "name": "productSet",
          "value": "my-set-id"
        }
      },
      "matchingRules": {
        "body": {
          "$.*": {
            "matchers": [{
              "match": "type"
            }],
            "combine": "AND"
          },
          "$.*.name": {
            "matchers": [{
              "match": "type"
            }],
            "combine": "AND"
          },
          "$.*.value": {
            "matchers": [{
              "match": "type"
            }],
            "combine": "AND"
          }
        }
      }
    }
  }],
  "metadata": {
    "pactSpecification": {
      "version": "3.0.0"
    },
    "pact-jvm": {
      "version": "4.0.8"
    }
  },
  "createdAt": "2024-04-21T13:47:08+00:00"
}

Cheers

[rholshausen]

4.1.x doesn't properly support the each-key matcher. Either upgrade your consumer to use 4.6.x and re-run the test.

If you can't upgrade to a 4.6 version, you can upgrade the consumer side to 4.1.42 and then set

FeatureToggles.toggleFeature(Feature.UseMatchValuesMatcher, true);
val providerResponseBody = PactDslJsonBody()
            .eachKeyLike("abcde")
            .stringType("name", "productSet")
            .stringType("value", setId)
            .closeObject()

in the consumer test. Note that this will generate a Pact file that can't be verified with a 4.1 version, but will pass ok with 4.6.

You should use the latest 4.6 version which is currently 4.6.9.