Export the release URL

[AlmogBaku]

Hi, it could be cool if you can export the release URL

[webknjaz]

It's an interesting idea. I'd be open to it.

Implementation hint:
Twine prints out the release URL so it'd need a wrapper that could parse it before exiting. We can't just guess it by concatenating the PyPI URL because it's not the only index implementation out there, and we shouldn't leave out deployments like devpi, pulp or various proprietary package repository implementations.

[thekaveman]

I have a POC of this sort of working, but I wanted to check in and see if it's the right direction and if you all had any early feedback. I would be happy to put together a PR with these changes if that would be more appropriate for feedback/review etc.

Here's a branch in my fork: unstable/v1...compilerla:gh-action-pypi-publish:feat/output-release-url

Changes

I modified twine-upload.sh like so:

+ TWINE_RESULT=$(
    TWINE_USERNAME="$INPUT_USER" \
    TWINE_PASSWORD="$INPUT_PASSWORD" \
    TWINE_REPOSITORY_URL="$INPUT_REPOSITORY_URL" \
-  exec twine upload ${TWINE_EXTRA_ARGS} ${INPUT_PACKAGES_DIR%%/}/*
+   twine upload ${TWINE_EXTRA_ARGS} ${INPUT_PACKAGES_DIR%%/}/*
+ )
+ TWINE_RETURN=$?
+
+ if [[ "$TWINE_RESULT" =~ View\ at:\ *\(https?://[^[:space:]]+\) ]]; then
+   RELEASE_URL="${BASH_REMATCH[1]}"
+   echo "release-url=$RELEASE_URL" >> "$GITHUB_OUTPUT"
+ fi
+
+ exit $TWINE_RETURN

action.yml also needs to be modified as follows:

---
name: pypi-publish
description: Upload Python distribution packages to PyPI
inputs:
...
+ outputs:
+  release-url:
+    description: >-
+      The URL on the package index of the newly uploaded package.

Summary

1. Remove usage of exec and wrap the twine upload call to capture its output
2. Look in the twine output for a string like

   View at:
   https://pypi.org/project/my_package/0.1.0/
   

3. If found, write the URL to $GITHUB_OUTPUT as the release-url value
4. release-url is available as an output of this action

Questions

1. Is twine-upload.sh the right place to do this?

   Twine prints out the release URL so it'd need a wrapper that could parse it before exiting.

   Did you mean another wrapper script? Or wrapping the command like shown above?

2. Does twine upload always output the URL like this? In my exploration, it seemed so -- except for the devpi server, which doesn't output any URL upon upload in my testing.

3. Is writing directly to $GITHUB_OUTPUT preferred? It is the documented way to set output parameters. But that variable won't exist e.g. locally, so the script may have to be further modified to account for other runtime environments.

Any other feedback would be greatly appreciated.

[webknjaz]

Thanks for posting the patch! I feel like it might be easier once we migrate the entry point script to Python. I've been meaning to stop accumulating more shell scripts for a while :)

Answers

1. Anything really. But refactoring in Python would be my preference.
2. No idea, but it seems you've found corner cases already..
3. Yes, it's preferred. This entire thing is designed to only run in GHA runtime so the variable presence is pretty much guaranteed. For local testing, it'd not be that hard to stick a GITHUB_OUTPUT=/dev/stderr in front of the command. However, I don't think that most of the bits here are testable locally even with that.

[flying-sheep]

This would be amazing, let’s please make this happen!

[webknjaz]

One other corner case that wasn't yet mentioned in the discussion is uploading multiple dists of different PyPI projects, possibly even having multiple different release numbers. All these would have different URLs. So said output would probably have to be serialized as a JSON list or something along the lines.

Alternatively, we could forbid releasing dists belonging to different PyPI projects or having different versions.

As for retrieving the URL, it's probably difficult until Twine has a proper API. There's years of debates about it: pypa/twine#194 / pypa/twine#361. Getting something usable and real into Twine would be a huge win and enable us to use it instead of the CLI with all the cool advantages like accessing attributes of things w/o having to pars stuff spit out onto the console...

[flying-sheep]

Since both sdist file names and wheel file names are standardized, it’d be easy to infer this information. As you said, rewriting the bash script to Python would probably be the way to go here (especially if sdist names/versions aren’t normalized, I didn’t check)

But I also agree about

Getting something usable and real into Twine would be a huge win

I just don’t know if it’s practical to wait for something there’s years of debate about.

[webknjaz]

@flying-sheep imagine you managed to special-case TestPyPI and PyPI. How do you do that for devpi/nexus/pulp/whatever? Perhaps, it'd be better to just return a mapping of normalized project names to the release version numbers… Or just a series of pairs, letting the end-users render URLs out of that.

Imagine I'm uploading the following

pkg_a-any-1.0.whl
pkg_a-3.1.tar.gz
pkg_b-any-3.0.whl
pkg_a-6.3.tar.gz

to devpi.

What would you put into the output called urls? And maybe into an output called releases?

[flying-sheep]

mapping of normalized project names to the release version numbers

to normalized release version numbers, yes!

[webknjaz]

So would the output look smth like

releases: >-
  {"pkg-a": ["1.0", "3.1"], "pkg-b": ["3.0", "6.3"]}

?

[flying-sheep]

I think that would make sense!