fix(test): force test failure

[oscard0m]

Description

Assure 'test' Check Run is always executed and fails/succeeds accordingly to the results of its dependency jobs.

• Example commit with all tests passing ✅: c4a2c50
• Example commit with tests failing 🔴: 8d93278

---

Context

When using job.needs, by default only runs when the dependency jobs succeed. If one or more of them fail, the dependent job is skipped

[oscard0m]

@gr2m @wolfy1339 @timrogers

I think I found the issue. I'm surprised this did not appear before.

What do you think about my solution proposal? Do you agree with solving this in the context of octokit/.github#13?

[timrogers]

What is your proposed solution? I'm not sure I understand from this PR.

[wolfy1339]

What is your proposed solution? I'm not sure I understand from this PR.

The solution proposed is to always run the test job that depends on the test_matrix jobs regardless of if they fail or not, in order to avoid having GitHub Actions skip it, and resulting in the skipped status.
That skipped status counts as if the tests passed for branch protection and allows us to merge PRs when they shouldn't.

In the test job, we check if the test_matrix jobs failed and then return an exit code of 1 in order to have it fail

[timrogers]

Sounds reasonable to me 🤷🏻

[wolfy1339]

What do you think about my solution proposal? Do you agree with solving this in the context of octokit/.github#13?

This will work for now, until we sort out the reusable workflows

[oscard0m]

This will work for now, until we sort out the reusable workflows

@wolfy1339
Do you mean you want me to apply this change to all our repositories until octokit/.github#13 is addressed?

[wolfy1339]

This will work for now, until we sort out the reusable workflows

@wolfy1339 Do you mean you want me to apply this change to all our repositories until octokit/.github#13 is addressed?

Yes. That would be great

[gr2m]

Change looks good, I think we did something similar in a different project.

[gr2m]

Here is the discussion we had in @semantic-release, we arrived at the same conclusion, but did not yet implement the change

semantic-release/semantic-release#2229