Use matchers when verifying metadata #745

mduesterhoeft · 2021-09-22T11:44:19Z

Feature description

Since v10.0.0-beta.45 metaData present in the consumer contract is verified.

It would be great if we could also use matches when verifying metadata.

Something like this should be working.

  "messages": [
    {
      "matchingRules": {
        "body": {},
        "metaData": {
          "$.traceId": { "combine": "AND", "matchers": [{ "match": "type" }] }
        }
      },
      "metaData": {
        "traceId": "8bf558a2-0b0d-457e-b1b2-75800352d637"
      }
    }
  ]

Use case

The metaData object contains a traceId which is change whenever the consumer contract is published. The verification should just make sure that the field exists and carries a string. A match by value is not useful.

The text was updated successfully, but these errors were encountered:

mefellows · 2021-09-22T11:56:32Z

Thanks @mduesterhoeft. I just took a quick look at the source and it seems matchers should be applied.

Could you please set the log level to trace and share an appropriately redacted log? That will help us diagnose what went wrong.

mefellows · 2021-09-22T11:59:36Z

Actually, on closer inspection that output is a bit confusing - metaData should actually be metadata.

The core knows how to read in both metaData and metadata at the interaction level, but not for the matching rules.

But it shouldn't actually be generating that. This might be a bug in Pact JS (not sure how yet) or in the version of the FFI we are using.

(I'd still love those logs if you could please share)

TimothyJones · 2021-09-22T12:07:03Z

Can you tell us which pact framework + version produced the pact file?

mduesterhoeft · 2021-10-11T16:08:07Z

Sorry for the confusion. I played around with the contract a bit - and posted a modified version. The contract the consumer generates has metadata in the matchingRules section - but metaData on interaction level.

So my snippet above is really this one:

"messages": [
    {
      "matchingRules": {
        "body": {},
        "metadata": {
          "$.traceId": { "combine": "AND", "matchers": [{ "match": "type" }] }
        }
      },
      "metaData": {
        "traceId": "8bf558a2-0b0d-457e-b1b2-75800352d637"
      }
    }
  ]

This is the version information from the contract ({ "pact-jvm": { "version": "4.2.3" }, "pactSpecification": { "version": "3.0.0" } })

I tried with 10.0.0-beta.48 on the provider side.

Here are the relevant pieces from the log

[2021-10-11 15:55:35.652 +0000] INFO: [email protected]: Verifying message
[2021-10-11 15:55:35.661 +0000] INFO: [email protected]: Verifying Pacts.
[2021-10-11 15:55:35.662 +0000] INFO: [email protected]: Verifying Pact Files
[2021-10-11 15:55:35.746 +0000] DEBUG: [email protected]: sending arguments to FFI:
[2021-10-11 15:55:35.746 +0000] DEBUG: [email protected]: --request-timeout
30000
--loglevel
trace
--provider-name
<provider>
--provider-version
1
--file
<local-contract-file>
--port
60293
--hostname
localhost

 RUNS   <testname>  <filename
  Given ...
    WARNING: State Change ignored as there is no state change URL provided
  Given ...
    WARNING: State Change ignored as there is no state change URL provided
  Event is triggered
    generates a message which
      includes metadata
        "traceId" with value "<actual-traceId-from-contract>" (FAILED)
      has a matching body (OK)


Failures:

1) ...
    1.1) has matching metadata
           Expected message metadata 'traceId' to have value '"<actual-traceId-from-contract>"' but was '"<trace-id-generated-by-provider>"'

So it seems to recognize the metaData from the interaction but compares against the real value from metaData instead of applying the matchingRules

NiklasEi · 2021-10-26T14:32:48Z

Is there an update on this? I am using pact-js to generate a MessageConsumerPact with meta data. The pact ends up containing a messages.metaData section with a regex matcher:

       "metaData": {
        "kafka_topic": {
          "json_class": "Pact::Term",
          "data": {
            "generate": "some-project-production-some-event",
            "matcher": {
              "json_class": "Regexp",
              "o": 0,
              "s": "some-project-(production|validation)-some-event"
            }
          }
        }
      }

On the provider side this fails with:

metadata: Expected metadata key 'kafka_topic' to have value '{json_class=Pact::Term, data={generate=some-project-production-some-event, matcher={json_class=Regexp, o=0, s=some-project-(production|validation)-some-event}}}' (LinkedHashMap) but was 'some-project-production-some-event' (String)

~~If I change metaData in the pact file to metadata the test is green.~~
Edit: I realized changing the pact file like this just caused the metadata to not be checked.

mefellows · 2021-10-26T23:08:31Z

Hi @NiklasEi, that pact file looks broken - it should not have json_class etc. in it. It's being generated by the current major version and not the v3 beta.

There have been issues with that in the past with the Ruby core, but please check you're on the latest version and have run a clean test.

NiklasEi · 2021-10-27T08:28:13Z

Thank you for the answer. Since this seems to be a different problem, I opened #763.

mduesterhoeft · 2021-11-10T13:19:43Z

@mefellows do you already have an idea on when you can fit this in. If you explain how you envision a solution I would also be happy to give it a try and come up with a PR.

mefellows · 2021-11-16T10:47:28Z

Thanks @mduesterhoeft, apologies for the delay in my response.

I don't have the answer right now I'm afraid. I just had a look through the rust source code and it seems any references to "metaData" have been addressed.

With the next release of the beta, I'm hoping this problem should be resolved.

cc: @uglyog as he may be able to provide a little more detail.

mefellows · 2021-11-16T10:48:27Z

In the meantime, i'd remove it from your consumer test until we fix the validation (which I assume you're probably doing!) or you could potentially post-process the pact file to replace metaData with metadata to see if it gets picked up properly.

NiklasEi · 2021-11-16T12:05:57Z

In the meantime, i'd remove it from your consumer test until we fix the validation (which I assume you're probably doing!) or you could potentially post-process the pact file to replace metaData with metadata to see if it gets picked up properly.

We tried it with metadata, but then the metadata is ignored. At the moment we just hardcode one value.

mefellows · 2021-11-16T12:10:15Z

👍

…

On Tue, 16 Nov 2021, 11:06 pm Niklas Eicker, ***@***.***> wrote: In the meantime, i'd remove it from your consumer test until we fix the validation (which I assume you're probably doing!) or you could potentially post-process the pact file to replace metaData with metadata to see if it gets picked up properly. We tried it with metadata, but then the metadata is ignored. At the moment we just hardcode one value. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAANFDGBHF3A6PHF6MZXUVTUMJCLBANCNFSM5ERDXNBA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

mduesterhoeft · 2022-05-18T12:56:34Z

With the next release of the beta, I'm hoping this problem should be resolved.

@mefellows Hi there - do you have any update for me on this topic? I am also totally ready to get my hands dirty to fix this issue with a PR from my side. But I would need an initial pointer, solution idea to get started.

mefellows · 2022-05-19T12:02:46Z

Thanks @mduesterhoeft. I'm not sure if it has been resolved.

You could

run a quick test on the most recent beta just to check if it's been fixed
if not, take a look around at https://github.com/pact-foundation/pact-reference/tree/master/rust. I haven't been in the rust core for a while, but @uglyog might be able to give you some advice (perhaps worth chatting with us in this channel)

mduesterhoeft · 2022-05-20T06:43:41Z

Thanks @mefellows

run a quick test on the most recent beta just to check if it's been fixed

I did that before bothering you again. I have the same issue with the latest beta.

If the solution would be in the rust core I am afraid I cannot help. I was hoping that this was in the js world still.

Kampfmoehre · 2022-08-04T10:58:16Z

Hi, I have a similar problem, I am using matchers in message consumer metadata like this

.withMetadata({
  contentType: "application/json", // Working
  correlationId: Matchers.string("nCB3FYqJIugLvQZ2b1dV") // Not Working
})

but the generated Pact file contains only the example value

{
  "consumer": {},
  "messages": [
    {
      "contents": {
        // ...
      },
      "matchingRules": {
        "body": {
          "$.name": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          }
        }
      },
      "metadata": {
        "contentType": "application/json",
        "correlationId": "nCB3FYqJIugLvQZ2b1dV",
      },
      "providerStates": [
        {
          // ...
        }
      ]
    }
  ]
}

This causes the provider tests to fail since the correlationId is a generated value that changes each time, the test output looks like this

    1.1) has matching metadata
           Expected message metadata 'correlationId' to have value '"nCB3FYqJIugLvQZ2b1dV"' but was '"YUqJ2pQhyYPqS5k1W5-5"'

Both use PactJS 10.0.1. I have tried MatchersV3 instead but that gives my TypeScript errors since the withMetdata method expects Matchers. Is there anything else I can provide to help with this?

TimothyJones · 2022-08-04T12:51:51Z

my TypeScript errors since the withMetdata method expects Matchers

This probably is a mistake in the pact v10 types - I think it would be more correct to give a V3 matcher there. However, I believe the string matcher is identical in both spec versions, so it's not the source of the problem (if you want to confirm it doesn't fix the problem, you can try as Matcher<string>).

Update: I think I found the issue:

https://github.com/pact-foundation/pact-js/blob/master/src/messageConsumerPact.ts#L178

Could you try patching that line in your node_modules? I'm guessing a bit, but I think it should say:

typeof v === 'string' ? v : JSON.stringify(v)

@mefellows if this does fix it, I think pact-js should just pass the whole thing down to the core, and the core should do the conversion.

Kampfmoehre · 2022-08-04T12:55:58Z

@TimothyJones I tested your line and Pact json now looks like this

{
        "correlationId": "{\"value\":\"nCB3FYqJIugLvQZ2b1dV\",\"pact:matcher:type\":\"type\"}"
}

TimothyJones · 2022-08-04T13:29:51Z

Ah 😅 apologies.

I think the next culprit is in the rust core. I’ll leave this for a maintainer to look at.

rholshausen · 2022-08-08T01:27:02Z

@Kampfmoehre That is wrong, there are only matching rules for the body

"matchingRules": {
        "body": {
          "$.name": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          }
        }
      },

rholshausen · 2022-08-08T01:27:46Z

The matching rules for the metadata are not being created in the Pact file.

mefellows · 2022-08-08T01:40:29Z

The matching rules for the metadata are not being created in the Pact file.

Does the rust core support matching rules on the metadata? I think that's the question.

rieckpil · 2022-12-07T14:27:23Z

Can anyone confirm that matching rules are supported for metadata? I'm struggling to get the matching rules for metadata for an asynchronous message (Pact V4) working. The Pact file properly inculdes matching rules for the metadata, e.g.:

{
  "consumer": {
    "name": "test"
  },
  "interactions": [
    {
      "matchingRules": {
         "metadata": {
          "Insert-Time": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          },
        },
        "metadata": {
          "Insert-Time": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          }
       }
   },
      "metadata": {
        "Correlation-Id": "bc0c3b06-2123-4824-a866-25767a060249",
        "Insert-Time": "2022-05-26T06:34:24.123Z",
        "Tenant-Id": "CIR7nQwtS0rA6t0S6ejd",
        "contentType": "application/json"
      },

But somehow the log states (running it with TRACE as the log level) there is no matching rule defined and it fall back to an equality check:

2022-12-07T14:07:34.829777Z TRACE ThreadId(02) pactffi_verifier_execute{handle=0x128e1d4d0}:verify_interaction{interaction="test"}: pact_models::matchingrules: matcher_is_defined: for category metadata and path ["Insert-Time"] -> false
2022-12-07T14:07:34.829780Z DEBUG ThreadId(02) pactffi_verifier_execute{handle=0x128e1d4d0}:verify_interaction{interaction="test"}: pact_matching::json: JSON -> JSON: Comparing '"2022-05-26T06:34:24.123Z"' to '"2022-12-07T14:07:34.807Z"' using Equality -> Err(Expected '2022-05-26T06:34:24.123Z' to be equal to '2022-12-07T14:07:34.807Z')

The Pact is created with Pact JVM 4.4.1 and the provider is using Pact-JS 10.2.2

mefellows · 2023-03-03T05:22:03Z

Could you please re-test? The latest version should support this now.

TezzM0 · 2023-11-22T05:22:47Z

This still seems to be happening for me with version 12.1.1.

When I include metadata in the pact like so:

      .withMetadata({
        'message-id': uuid("a0d38a09-f2da-46cf-8267-3cd575056606"),
        'message-type': 'create-product',
      })

It generates a pact contract that looks like this:

{
      "contents": {
        "body": {
          "createdAt": "2015-08-06T16:53:10.123+01:00",
          "id": "6551bfe2a3168487de1773dd",
          "updatedAt": "2015-08-06T16:53:10.123+01:00"
        },
        "headers": {
          "message-id": "5892a89d-c665-4dc2-939a-8a0d1f7744cc",
          "message-type": "create-product"
        }
      },
      "description": "A product created event",
      "matchingRules": {
        "body": {
          "$.body": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          },
          "$.body.createdAt": {
            "combine": "AND",
            "matchers": [
              {
                "match": "regex",
                "regex": "^\\d{4}-[01]\\d-[0-3]\\dT[0-2]\\d:[0-5]\\d:[0-5]\\d\\.\\d+([+-][0-2]\\d(:?[0-5]\\d)?|Z)$"
              }
            ]
          },
          "$.body.id": {
            "combine": "AND",
            "matchers": [
              {
                "match": "regex",
                "regex": "^[a-fA-F0-9]{24}$"
              }
            ]
          },
          "$.body.updatedAt": {
            "combine": "AND",
            "matchers": [
              {
                "match": "regex",
                "regex": "^\\d{4}-[01]\\d-[0-3]\\dT[0-2]\\d:[0-5]\\d:[0-5]\\d\\.\\d+([+-][0-2]\\d(:?[0-5]\\d)?|Z)$"
              }
            ]
          },
          "$.headers": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          },
          "$.headers['message-id']": {
            "combine": "AND",
            "matchers": [
              {
                "match": "regex",
                "regex": "^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$"
              }
            ]
          }
        }
      },
      "metadata": {
        "contentType": "application/json",
        "message-id": "a0d38a09-f2da-46cf-8267-3cd575056606",
        "message-type": "create-product"
      }
    },

The header matching rules are now included in the pact, which I didn't see happening previously (not sure which version I last tried with, sorry) - but they aren't used by the provider during verification:

A product created event (1s 994ms loading, 3ms verification)
    generates a message which
      includes metadata
        "message-type" with value "create-product" (OK)
        "message-id" with value "a0d38a09-f2da-46cf-8267-3cd575056606" (FAILED)
        "contentType" with value "application/json" (OK)
      has a matching body (OK)

Pending Failures:
1) Verifying a pact between repeat-rewards and product-catalogue - A product created event
    1.1) has matching metadata
           Expected message metadata 'message-id' to have value '"a0d38a09-f2da-46cf-8267-3cd575056606"' but was '"569079cd-611c-47a0-9968-ddf88ac6badc"'

It still requires the UUID to match exactly, rather than using the regex specified in the matchers.

github-actions · 2023-11-23T23:00:44Z

🤖 Great news! We've labeled this issue as smartbear-supported and created a tracking ticket in PactFlow's Jira (PACT-1495). We'll keep work public and post updates here. Meanwhile, feel free to check out our docs. Thanks for your patience!

mefellows · 2023-11-29T22:43:48Z

I think this would be resolved via this change: https://github.com/pact-foundation/pact-reference/pull/343/files

ferrisbuhler · 2024-02-26T17:29:27Z

I checked with pact-js 12.1.2 and 12.2.0. For both versions

MessageConsumerPact refuses to accept V3 matchers (such as MatchersV3.uuid() in metadata (see (TypeScript) Cannot use V3 matchers within MessagePact. Types are incompatible. #1133)
no matchers are generated in the "Pact" contract for the metadata.

Any news on this? Any workarounds to be recommended?

Supports metadata with matching rules Fixes pact-foundation/pact-js#1133 Fixes pact-foundation/pact-js#745

Fixes #1133 Fixes #745

mefellows · 2024-02-28T04:46:11Z

This should be fixed now - thanks all.

mduesterhoeft added the enhancement label Sep 22, 2021

mefellows added the Awaiting feedback label Sep 22, 2021

mefellows added the bug Indicates an unexpected problem or unintended behavior label Sep 22, 2021

rholshausen mentioned this issue Jan 9, 2023

Matching rules are not being applied to message metadata pact-foundation/pact-reference#245

Closed

mefellows removed Awaiting feedback labels Jul 12, 2023

mefellows added the smartbear-supported SmartBear engineering team will support this issue. See https://docs.pact.io/help/smartbear label Nov 23, 2023

mefellows added a commit to pact-foundation/pact-js-core that referenced this issue Feb 27, 2024

feat: add pactffi_message_with_metadata_v2

bd6e256

Supports metadata with matching rules Fixes pact-foundation/pact-js#1133 Fixes pact-foundation/pact-js#745

mefellows mentioned this issue Feb 28, 2024

feat: add pactffi_message_with_metadata_v2 pact-foundation/pact-js-core#493

Merged

mefellows closed this as completed in pact-foundation/pact-js-core#493 Feb 28, 2024

mefellows added a commit to pact-foundation/pact-js-core that referenced this issue Feb 28, 2024

feat: add pactffi_message_with_metadata_v2 (#493)

613b7bd

Supports metadata with matching rules Fixes pact-foundation/pact-js#1133 Fixes pact-foundation/pact-js#745

mefellows added a commit that referenced this issue Feb 28, 2024

chore: allow v3 matchers in message metadata

2085b45

Fixes #1133 Fixes #745

mefellows added a commit that referenced this issue Feb 28, 2024

chore: allow v3 matchers in message metadata

2eac5f3

Fixes #1133 Fixes #745

mefellows added a commit that referenced this issue Feb 28, 2024

chore: allow v3 matchers in message metadata

93d0fa8

Fixes #1133 Fixes #745

mefellows mentioned this issue Feb 28, 2024

chore: allow v3 matchers in message metadata #1183

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use matchers when verifying metadata #745

Use matchers when verifying metadata #745

mduesterhoeft commented Sep 22, 2021

mefellows commented Sep 22, 2021

mefellows commented Sep 22, 2021 •

edited

Loading

TimothyJones commented Sep 22, 2021

mduesterhoeft commented Oct 11, 2021 •

edited

Loading

NiklasEi commented Oct 26, 2021 •

edited

Loading

mefellows commented Oct 26, 2021

NiklasEi commented Oct 27, 2021 •

edited

Loading

mduesterhoeft commented Nov 10, 2021

mefellows commented Nov 16, 2021

mefellows commented Nov 16, 2021

NiklasEi commented Nov 16, 2021

mefellows commented Nov 16, 2021 via email

mduesterhoeft commented May 18, 2022

mefellows commented May 19, 2022

mduesterhoeft commented May 20, 2022 •

edited

Loading

Kampfmoehre commented Aug 4, 2022

TimothyJones commented Aug 4, 2022

Kampfmoehre commented Aug 4, 2022

TimothyJones commented Aug 4, 2022

rholshausen commented Aug 8, 2022

rholshausen commented Aug 8, 2022

mefellows commented Aug 8, 2022

rieckpil commented Dec 7, 2022 •

edited

Loading

mefellows commented Mar 3, 2023

TezzM0 commented Nov 22, 2023

github-actions bot commented Nov 23, 2023

mefellows commented Nov 29, 2023

ferrisbuhler commented Feb 26, 2024 •

edited

Loading

mefellows commented Feb 28, 2024 •

edited

Loading

Use matchers when verifying metadata #745

Use matchers when verifying metadata #745

Comments

mduesterhoeft commented Sep 22, 2021

Feature description

Use case

mefellows commented Sep 22, 2021

mefellows commented Sep 22, 2021 • edited Loading

TimothyJones commented Sep 22, 2021

mduesterhoeft commented Oct 11, 2021 • edited Loading

NiklasEi commented Oct 26, 2021 • edited Loading

mefellows commented Oct 26, 2021

NiklasEi commented Oct 27, 2021 • edited Loading

mduesterhoeft commented Nov 10, 2021

mefellows commented Nov 16, 2021

mefellows commented Nov 16, 2021

NiklasEi commented Nov 16, 2021

mefellows commented Nov 16, 2021 via email

mduesterhoeft commented May 18, 2022

mefellows commented May 19, 2022

mduesterhoeft commented May 20, 2022 • edited Loading

Kampfmoehre commented Aug 4, 2022

TimothyJones commented Aug 4, 2022

Kampfmoehre commented Aug 4, 2022

TimothyJones commented Aug 4, 2022

rholshausen commented Aug 8, 2022

rholshausen commented Aug 8, 2022

mefellows commented Aug 8, 2022

rieckpil commented Dec 7, 2022 • edited Loading

mefellows commented Mar 3, 2023

TezzM0 commented Nov 22, 2023

github-actions bot commented Nov 23, 2023

mefellows commented Nov 29, 2023

ferrisbuhler commented Feb 26, 2024 • edited Loading

mefellows commented Feb 28, 2024 • edited Loading

mefellows commented Sep 22, 2021 •

edited

Loading

mduesterhoeft commented Oct 11, 2021 •

edited

Loading

NiklasEi commented Oct 26, 2021 •

edited

Loading

NiklasEi commented Oct 27, 2021 •

edited

Loading

mduesterhoeft commented May 20, 2022 •

edited

Loading

rieckpil commented Dec 7, 2022 •

edited

Loading

ferrisbuhler commented Feb 26, 2024 •

edited

Loading

mefellows commented Feb 28, 2024 •

edited

Loading