Add strict checking of AWS API mock endpoints.

[theipster]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

References: #369. (This PR updates the existing test suite to assert on mock API counts. These count assertions can then be used to replicate the scenario mentioned in #369.)

---

MR description

Previously, an unordered list of endpoint definitions were provided to the mock API server and incoming requests were matched with a simple existence lookup.

Now:

• the list is ordered and requests are matched in FIFO order (i.e. any out-of-sequence request will be returned a HTTP 400); and,
• matched requests will be shifted off the list (to avoid being matched multiple times); and,
• every test scenario will additionally check that the list of expected endpoints for that scenario is fully exhausted.

Ultimately, the result is that each test scenario must exactly match the list of expected endpoints - no more and no less.

The objective is to expose duplicated API calls that are not only inefficient, but also potentially cause the API service to rate-limit (see hashicorp/terraform-provider-aws#27071).

[theipster]

The corollary / pièce de résistance lies in the mock endpoints here: https://github.com/theipster/aws-sdk-go-base/pull/3/files#diff-ffb4ec9fa0ecffd143689744bed517bd2de0400060c2c0855755ab247dbfd94fR1244-R1257.

You can see that the servicemocks.MockStsAssumeRoleWithWebIdentityValidEndpoint mock endpoint is invoked once per provider, whereas it could theoretically be cached given that the inputs come from environment variables only.

Scale this up to a non-trivial number of providers, and this is where AWS STS starts throttling requests (as per hashicorp/terraform-provider-aws#27071).