Github Actions raises a 422 error

[TheKevJames]

This is a known issue -- there are some cases where Github actions needs to be run with the github-actions service name and some cases where it must use the github service name. We have as of yet been unable to determine what causes the difference in usecase.

Running coveralls on Github Actions will use the github-actions service name by default. If you see a 422 error get raised, please try running coveralls --service=github, instead:

coveralls --service=github

If anyone discovers the secret sauce behind when coveralls requires each name, please let me know -- I'd love to fix the auto-detection!

[vchrombie]

Hi @TheKevJames, thanks for your work with this tool.

I was following up on the recent updates to this tool. I came across this workaround and I was trying to use this.

I was trying to migrate a few repositories from travis to github actions and I faced the 422 error even after using the coveralls --service=github. Strange is that it worked in one of the similar repos earlier but not working in this.

👍 grimoirelab-perceval-opnfv build and its workflow file
👎 grimoirelab-perceval-mozilla build and its workflow file

[andy-maier]

Just in case the additional information helps: In our projects, the 422 error only shows up on Python 3.9 with latest package levels. Not on Python 3.9 with our minimum package levels, and not on older Python versions. See for example this Actions run: https://github.com/pywbem/pywbem/actions/runs/484671222

[liskin]

@TheKevJames Do we have some examples of situations where github-actions service name is needed? Most of the commits linked to this issue are switching back to github, and that is what we'll need to do in taskwiki as well. The tests there use the GITHUB_TOKEN provided by Actions, no COVERALLS_REPO_TOKEN, and there is COVERALLS_PARALLEL=true. With github-actions service name, we get 422.

I was involved in #227 which made it possible to do test matrix builds with coveralls correctly, so I might be able to remember something, but my memory of it is somewhat foggy.

(I also think that service_job_id only needs to be specified with github-actions service name. With github, it's worked just fine without it, as it should: GH Actions don't provide any unique GITHUB_JOB_ID. It is probably harmless to pass null, though.)

[liskin]

@andy-maier

Just in case the additional information helps: In our projects, the 422 error only shows up on Python 3.9 with latest package levels. Not on Python 3.9 with our minimum package levels, and not on older Python versions. See for example this Actions run: https://github.com/pywbem/pywbem/actions/runs/484671222

That's because python 3.9 + latest package levels is the only configuration that actually uses the new coveralls-python 3.0.0. All the other use 2.1.2, which isn't broken by f4faa92 yet.

[liskin]

@nickmerwin Would you perhaps be able to shed some light on what's the difference between github and github-actions service names from the coveralls API point of view? The documentation (https://docs.coveralls.io/supported-ci-services and https://docs.coveralls.io/api-reference) is very hand-wavy on this: “This can be anything, but certain services have special features (travis-ci, travis-pro, or coveralls-ruby).” :-/

[TimoRoth]

My understanding of it is that if you are using the Github Actions natively provided GITHUB_TOKEN secret or ${{ github.token }}, you use the github service. Coveralls then works some internal magic, querying build and authentication info via GitHub APIs using that token.
If you manually provide your coveralls token via a secret, you use github-actions (or really anything that's not a special-cased service name).

[Ri-Dearg]

Hi, thank you so much for this great package.
I recently found myself facing this issue after updating to 3.0.0.
I set a natively provided GITHUB_TOKEN secret.
I didn't set any --service argument while using Github Actions in version coveralls 2.2.0 and it used to work just fine without one.
After the upgrade to 3.0.0, I am facing the error 422 with no argument.
Now I must manually set --service=github to successfully deploy the report. --service=github-actions fails. This would imply that @TimoRoth is correct.
I don't know if that's any help to you in figuring this out. I ran a few tests to confirm, if you want to see the history, take a look here.

[kennedykori]

@stephane-caron solution worked for me too.

[leeqvip]

I solved it this way, removing the repo from https://coveralls.io/ and adding it back.

[joshua-goldstein]

I also ran into this issue in the opposite way (I was using the --service=github flag and was getting an error). The issue was resolved by removing this flag.

[elParaguayo]

This has just started out of the blue for us.

I thought it might have been linked to an upgrade to coverage as the timings worked but downgrading that package didn't fix the issue.

We had been using the --service=github flag. I've tried changing to github-actions and removing it completely. Neither worked.

I'd really prefer not to have to delete the data from coveralls. Is that really the only option here?

[andy-maier]

Since a few days, coveralls fails to upload from GitHub Actions with the 422 error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs

This happened now also with coveralls 2.1.2, I retried with coveralls 3.3.1 with and without --service=github, butnone of that worked.

This happens on all Python versions we support (2.7, 3.5-3.11).

This started occurring only a few days ago. Before that, it ran fine.
The latest failing run with my attempts to fix it is: https://github.com/zhmcclient/python-zhmcclient/actions/runs/4839303086
The last run where it still ran fine is from 4/23 and is: https://github.com/zhmcclient/python-zhmcclient/actions/runs/4774941292

I could not identify any relevant package version changes between the last run that worked and now.

Update:

I ran coveralls in debug mode, and for Python 3.11 with latest package levels (coveralls version 3.3.1),and with --service=github, and with GITHUB_TOKEN envvar set to secrets.GITHUB_TOKEN (i.e. run https://github.com/zhmcclient/python-zhmcclient/actions/runs/4842932176/jobs/8630194117), the output is:

{
  "source_files": [  (>30000 lines, omitted) ],
  "git": {
    "branch": "andy/fix-coveralls",
    "head": {
      "id": "b611647272966ebda04019f6852dba6b1ef3a964",
      "author_name": "Andreas Maier",
      "author_email": "[email protected]",
      "committer_name": "GitHub",
      "committer_email": "[email protected]",
      "message": "Merge b4723c0593e9ceab43a1a9be642203970dae992a into a11275d3d7b34acbb475eb7ba3d081fdb8c140be"
    },
    "remotes": [
      {
        "name": "origin",
        "url": "https://github.com/zhmcclient/python-zhmcclient"
      }
    ]
  },
  "service_job_id": null,
  "repo_token": "[secure]",
  "service_name": "github",
  "service_number": "4842932176",
  "service_pull_request": "1138",
  "parallel": true,
  "flag_name": "ubuntu-latest,3.11,latest",
  "config_file": ".coveragerc",
  "base_dir": "",
  "src_dir": ""
}

Note that service_job_idis None/null. Is that related to issue #384 ?

I have reported another occurrence of this issue to coveralls.io: lemurheavy/coveralls-public#1710

[andy-maier]

@elParaguayo Note that I removed and added one of our repos (https://github.com/zhmcclient/python-zhmcclient) from coveralls.io, and that did not fix the issue with this new occurrence of HTTP 422.

[andy-maier]

@TheKevJames One question: Is the JSON shown when running coveralls in debug mode, directly the HTTP body that is sent to coveralls.io?

Update: After looking at the code, I can answer this myself with : Yes.

[andy-maier]

I think I have found the fix for this issue, and have launched PR #385 with the fix.

For users finding this issue: Without using that PR, the same solution can be achieved in anyone's GitHub Actions workflow as follows, as long as a version of coveralls is used that supports the COVERALLS_SERVICE_JOB_ID env var (such as coveralls>=3.3.0):

For sending the result of each parallel run:

    - name: Send coverage result to coveralls.io
      shell: bash -l {0}
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        COVERALLS_PARALLEL: true
        COVERALLS_SERVICE_JOB_ID: ${{ github.run_id }}
      run: |
        coveralls --service=github

For sending the finish request for parallel runs:

    - name: Send coverage finish to coveralls.io
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: |
        coveralls --service=github --finish

If this solution does not work for you, or if you find that you have to use the service name "github-actions", please comment on this issue.

[spyoungtech]

Had no problems with coveralls on Sep 28 and then subsequently, Nov 10, it stopped working with this 422 error. The first job report succeeds, but whenever a subsequent parallel job tries to post the coverage report, it fails with 422. Something seems to have changed between those dates.

In any case, the suggestion by @andy-maier fixed the issue.

# ... matrix job
       - name: Coveralls
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          COVERALLS_PARALLEL: "true"
+          COVERALLS_SERVICE_JOB_ID: ${{ github.run_id }}
         run: |
           pip install --upgrade coveralls
           coveralls --service=github
+  finish:
+    runs-on: ubuntu-latest
+    needs: build
+    steps:
+      - name: finish coveralls
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          pip install --upgrade coveralls
+          coveralls --service=github --finish

[TheKevJames]

Default will be changing to --service=github in the next major release, coming shortly. Closing this thread now, as things should work a bit more solidly out of the box!