How to cut tagged releases from the default repo branch #42
-
|
I am a bit unsure if I understand what this does. So I am going to give an example for a workflow and explain what I expect this is doing. If this is correct you could include this example in your readme (I can create a pull request if you want). If this is false please correct me: name: publish
on:
push:
branches:
- master
# the following would create a release on any tag creation not necessarily a tag
# referring to a commit on master (i.e. tagged commits in branches would cause
# releases)
# tags:
# - 'v*'
jobs:
pypi:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
uses: actions/setup-python@v1
with:
python-version: 3.8
- name: build package
run: python setup.py sdist bdist_wheel
- name: publish to PyPI
# only create relases from tagged commits to master:
if: startsWith(github.ref, 'refs/tags')
uses: pypa/[email protected]
with:
user: __token__ # use pypi token instead of username/password
password: ${{ secrets.pypi_token }} # use secret from github secretsand another question: can I somehow restrict tagnames to |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 11 replies
-
|
apparently I can not read: I am still unsure on how it ensures only tags on master are used. But this cleared a lot of things up |
Beta Was this translation helpful? Give feedback.
-
|
It appears that
I woder if |
Beta Was this translation helpful? Give feedback.
-
A tag is a reference to a specific reference (like a commit), and is not associated with a branch. This SO answer provides a really good summary of how git branches and tags work. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the comment, I'm going to post a more complete explanation in the context of how GitHub App triggers work relating to how Git works. P.S. With discussions we now have threads. So it looks like it'd be better to post something like this as a reply to the thread: #42 (comment). |
Beta Was this translation helpful? Give feedback.
-
|
@pradyunsg I am aware that tags are associated with commits. That is why I used this awkward formulation:
i.e. wouldn't a trigger And I am not quite sure how to implement that yet. I mean a simple "on push" would alow anyone to create tons of pull requests which would immediately result in pushes to pypi wouldn't it? |
Beta Was this translation helpful? Give feedback.
-
|
Only folks with push access to a repository can push tags to it. This means, I can push a tag to pip's repository, but if you make a fork and push a tag there, it'll trigger the build over on your fork, which likely won't have the secrets (password/API token etc) to succeed.
You'd need to write logic to abort a build if the tag's commit is not on the branch. There's probably a git + bash mix that makes it possible to detect this. I'm not sure if/when you would need something like this -- is this a specific concern you're trying to resolve? |
Beta Was this translation helpful? Give feedback.
-
|
@pradyunsg there is no specific concern - but it is not a particularly elegant solution to let pipelines crash if they do not have the permissions. Additionally it also means I have to be careful with tagging commits on branches. I would prefer if pipelines were not even started in that case. What do you think about the "on release" idea? |
Beta Was this translation helpful? Give feedback.
-
|
The name: publish
on:
release:
types:
- "published"
jobs:
pypi:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
uses: actions/[email protected]
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: build package
run: python setup.py sdist bdist_wheel
- name: publish to PyPI
# only create relases from tagged commits to master:
if: startsWith(github.ref, 'refs/tags')
uses: pypa/[email protected]
with:
# if no "user:" specified use pypi token instead of username/password
password: ${{ secrets.pypi_token }} # use secret from github secrets |
Beta Was this translation helpful? Give feedback.
-
|
This way the Workflow also is named after the release name |
Beta Was this translation helpful? Give feedback.
-
|
Honestly, this way has the same flaws because hitting a release button is also uncoupled from testing and thus also relies on a human factor heavily. I prefer testing what I'm about to publish and only then publishing it. But so far I only have one project that implements this idea fully (it's still tag-triggered though). Besides, the release creation also creates a Git tag so it's practically the same. I'll try to publish a more complete answer about various considerations to be made. |
Beta Was this translation helpful? Give feedback.
-
|
@FelixBenning I tried making my answer more comprehensive and generic: #42 (comment). Let me know what you think! |
Beta Was this translation helpful? Give feedback.
-
|
@webknjaz I still think this is a good solution because it decouples the problem from pypi. As you have argued, matching tags to a branch is quite complicated. So one could separately solve this problem and use the github api to trigger releases with this solution. The pypi publication then piggybacks on this solution. So at this point it is not a pypi problem anymore but a: "how can I couple merges of merge requests with tagging and release creation?"- problem. And when thinking about this problem I realized: If I do not create tags automatically on merges (which is difficult as you then need an algorithm for naming these tags), you are going to have one additional human action anyway (in addition to the merge of the merge request). So if you have this additional action, you might as well click the release button instead of tagging a commit. What's more the release button has a nicer ui than the tag creation anyway. But if you actually want to create tags automatically when you merge a pull request, then you can solve this problem separately (but use the release api instead of just creating a tag) and my suggested pypi-release-trigger piggybacks along, as it is then triggered as a direct result. So in that sense it does what I want in both cases (manual tags/releases and automatic tags/releases). The number of total human actions required is either:
both are acceptable in my opinion. And since you use the release api to create these "release tags", you also have the option to create non-release tags (not sure if that ever becomes useful). Which you need to avoid if you trigger on every tag. (That is why I asked whether it was possible to filter tags and only use EDIT (regarding testing): in principle you already had tests running before you approved the merge request (in fact you can force green pipelines for merges I think). So whatever is in main/master is already tested so you can savely create releases from it. But if you are very paranoid you can just let your tests trigger on releases as well and wait with the pypi release until they finish. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, the point about a nicer UI is totally valid. In fact, I've been experimenting with using a GitHub App that would add a Release button to commits via Checks API and then trigger Deployments API on click (deployment events can also be used as triggers) but that didn't go further than private experiments. Regarding automatic publishing on merge, I have a number of suggestions about it. First of all, you could use a FWIW the actual solution to this problem really depends on the different needs every project may have. |
Beta Was this translation helpful? Give feedback.
-
The problem statementI take it you want to ensure that the tags causing your dist to be published "belong" to the main branch in your repository. This does sound straightforward high-level but the actual implementation may be rather complicated. And here's why. How stuff worksFirst of all, there are multiple technical sides to this puzzle. Let's explore each one of them. GitIn Git, tags are roughly pointers to commits. Branches are also labels on commits. There is absolutely no direct relationship between these two except that they both can be resolved to commit hashes. So just by knowing a tag name and a branch name, there's zero chance to figure out if that tag "belongs" to a given branch. It is possible to establish, though, if a commit the tag is associated with belongs to a branch. This means that in order to know if a tag is counted as pointing to something on a branch programmatically, we need a copy of the Git repo tree. One more thing, when the true merge commits are used, you'll see forks with subpaths in commit-tree graphs. It may come as a surprise but all of the commits that may seem to be belonging to another branch would also belong to the main branch upon merging. Let's look at this example: E * <- (branch: main)
|
D * <- [merge commit -- it has two parents: C and H]
|\
C * * H
| * G <- (tag: v1.0.0)
B * * F
|/
A *In this graph, commits Some projects use release-PRs: they make all necessary final release preparations in a branch their upstream repositories, then create a PR and tag the head of that branch. Events on the GitHub platformWhenever something happens on GitHub, like actions performed by users or integrations, they are exposed as events that can be analyzed or reacted upon programmatically. There's Events API and webhooks for this — one can either poll GitHub for what's happened or subscribe for events to be delivered to their server by GitHub. Now, let's take a look at what various events mean (amongst those related to our current needs):
GitHub Actions workflow triggersGitHub Actions workflows are built on top of GitHub Apps and the idea of subscribing to certain events that GitHub platform emits, including but not limited to the ones I mentioned above, and then firing up whatever scenarios you have defined. One workflow may subscribe to multiple events allowing to reuse the same collection of jobs and steps. You can also make some jobs and steps conditional and use event names and their context information to filter out things you don't want to run in some cases. Some of the event triggers ( There's many approaches to deciding how to trigger building and publishing automatic releases. Although, I think that the best semantically correct event to use for tag-based triggers would be if: github.event_name != 'create' || github.ref_type == 'tag'The last variant of trigger I want to talk about is Oh, and I want to emphasize that you can combine various triggers in order to create an automation that suits your needs best. Huh?Yeah, this probably sounds complicated but it is what it is. Based on the above, here's a final advice:
|
Beta Was this translation helpful? Give feedback.
-
|
good writeup - I finally understood the difference between I still prefer the |
Beta Was this translation helpful? Give feedback.
-
|
@FelixBenning every project's needs/preferences are different. So I wanted to have a more generic explanation of the topic. I'm glad that you've figured out what works for you but it seems like you've raised several related (but still different) questions and so I'm not sure if all the discussions should be merged. I think I'd prefer having good Q&A discussion topics posted and explained separately StackOverflow-style rather than dumped in one place because it may be confusing to the people who may come here searching for specific answers in the future... It'd be great if you decide to contribute more questions (and maybe answers) that are distinct, though. But please make sure to start separate discussions for each of them so that we could have a great knowledge base one day :) |
Beta Was this translation helpful? Give feedback.
The problem statement
I take it you want to ensure that the tags causing your dist to be published "belong" to the main branch in your repository. This does sound straightforward high-level but the actual implementation may be rather complicated. And here's why.
How stuff works
First of all, there are multiple technical sides to this puzzle. Let's explore each one of them.
Git
In Git, tags are roughly pointers to commits. Branches are also labels on commits. There is absolutely no direct relationship between these two except that they both can be resolved to commit hashes.
So just by knowing a tag name and a branch name, there's zero chance to figure out if that tag "belongs" to a given b…