[Associated vote ended on 2022-10-26] Collections that don't tag releases #148
Comments
|
Currently, the guidelines say the following about tagging:
I think this should be made stricter and more explicit. I will propose better wording later when I'm less tired :). I'll also put the data somewhere where others can look at it. |
|
I think that https://github.com/CiscoDevNet/ansible-nso (cisco.nso) might be unmaintained: The last commits, releases or CI runs are more than 1 year old. So maybe we should deprecate this collection. I've already found this one in #128 but I think I haven't done anything about this yet. |
gotmax23
added
discussion_topic
next_meeting
|
Here is my proposed update:
|
|
Here is the data: It includes the repository URLs and |
|
We discussed this in today's Community meeting. I'll create a guidelines PR and start a vote in a few days if there's no major objections to the new wording. After that, I'll create issues against each collection that violates this rule. @mariolenz proposed amending the collection removal policy to explicitly state that collections with unresolved guidelines violations are considered unmaintained. @Andersson007 suggested that the guidelines should recommend collections not under the community namespace to also the Zuul release pipeline to hopefully prevent issues like this. Eventually, I'd like to add collections' repository URLs and version tag formats to ansible-build-data and have an automated check to make sure the releases are tagged. We could also have antsibull-build put a file in the ansible package with links to the full source of each collection. That's probably a separate discussion, but I'm jotting it down here so I don't forget. This was very briefly discussed in the meeting. |
Is that a new name for the community meeting? :) I'm +1 to @mariolenz's suggestion. I'm also +1 to more closely linking collections and collection source repositories. For that, we probably have to restrict the allowed SCMs though. I guess we should first stricten the guidelines to require git repositories (maybe more SCM types later, but git suffices right now because that's what everyone is using), and then start implementing something. |
/me is in too many meetings :).
Currently, every collection uses Github except openstack.cloud which uses the opendev.org Gitea instance. Both Github and Gitea (used by opendev.org), have API endpoints to get a list of tags. I don't think it's likely that new collections will use mercurial or svn or something. I think we can cross that bridge if/when we come to it. Implementing support for Github, Gitea, and Gitlab is a good start. We could implement a fallback that actually clones a git repository and extracts the tags, but that should be a last resort, as it would be quite slow. I'm also not sure we need to implement the fallback until a new collection actually starts using another git forge. My main question is whether this check should be integrated in antsibull or somewhere else and how the data file should be formatted/stored in ansible-build-data. |
|
I don't see why we would need special support for GitHub, GitLab, or Gitea. Why not simply support git? git ls-remote --help
git ls-remote --tags https://opendev.org/openstack/ansible-collections-openstack.git
git ls-remote --tags https://github.com/ansible-collections/community.rabbitmq.git |
On the one hand, I agree since those are just different "git services". On the other hand, we require that all collections MUST have a public SCM repository. Actually, most people will use git nowadays but in theory, they could use any obscure SCM as long as the repo is publicly available. This question might be a bit out of scope for this issue, but should we require a public git repository? Instead of a (generic) SCM repository? |
|
@mariolenz I'm for requiring that one, and mentioning that we can potentially add more SCMs to the list once our tooling supports them. Not that anything gets the impression that we're making random requirements here :) |
I did not know about |
For folks who aren't subscribed to both repositories, @mariolenz created ansible-collections/overview#218 to require this. |
|
Updated suggestion to avoid having two discussions / PRs / votes:
- - [ ] have a public SCM repository, releases of the collection are tagged in this repository
+ - [ ] have a public git repository, releases of the collection are tagged in this repository
Repository management
=====================
- Every collection MUST have a public SCM repository and releases of the collection MUST be tagged in this repository.
+ Every collection MUST have a public git repository. Releases of the collection MUST be tagged in said repository. This means that releases MUST be ``git tag``ed and that the tag name MUST exactly match the Galaxy version number. Tag names MAY have a ``v`` prefix, but a collection's tag names MUST have a consistent format from release to release.
+
+ Additionally, collection artifacts released to Galaxy MUST be built from the sources that are tagged in the collection's git repository as that release. Any changes made during the build process MUST be clearly documented so the collection artifact can be reproduced.
+
+ We are open to allowing other SCM software once our tooling supports them. |
|
+1 to the new proposal. |
@gotmax23 I've re-opened ansible-collections/overview#218 and merged your suggestion. If there are more things you want to change, you could add them there. Or, if you prefer, I could close it again and you open a new one. If it's OK for you now and you don't want to change anything else, I think we should open a vote on this. |
mariolenz
added
active-vote
mariolenz
changed the title
mariolenz
changed the title
|
xref to the vote: #153 |
|
I counted votes: 8 x +1 from SC (mariolenz gotmax23 briantist felixfontein Andersson007 markuman russoz acozine), no vote from community this time. Can someone from the Steering Committee please confirm? |
I've checked and confirm the result: 8 SC x +1, no one is against |
|
In that case the proposal has been accepted. Can someone please approve / merge ansible-collections/overview#218? Thanks all! |
|
closed via ansible-collections/overview#218 , thanks everyone! |
|
I've reopened this to track filing issues against the seven offending collections. If someone really wants that to be a separate issue, let me know, and I'll create one. |
|
I plan on announcing this on the Bullhorn and maybe in https://github.com/ansible-collections/news-for-maintainers/. For the collections that violate this, what should the maintainers be asked to do? I was thinking of asking them to retroactively tag the last release or two. What do you think? |
The Bullhorn sounds like a good place to announce this an remember collection maintainers to tag their releases. I'm not sure about https://github.com/ansible-collections/news-for-maintainers/ though. Maybe @Andersson007 or @felixfontein could say if this is a good idea. Besides, just to make sure you might want to open issues in the problematic repos that they should tag their releases. Although not in mellanon.onyx since it looks unmaintained and will be removed in Ansible 8(?), anyway.
I think the minimum would be to tag the latest / current release. Of course, it would be nice if people would tag all past releases but the important thing is they tag all future ones. |
|
I would ask them to at least tag the last 1-2 releases, though I personally would prefer if they tag all. Probably also depends on how many they had, if they only had very few it shouldn't be much work anyway. About news-for-maintainers: I would only announce there a general reminder to do this, but I don't think it's necessary if we inform the collections who violate this directly. |
Yes, the plan was to copy and paste the same issue for each repository. I'll skip mellanox.onyx, but I don't think it would hurt to file an issue for that one; we'd just have more evidence that it's unmaintained.
Me too. I'll ask maintainers to tag at least the last two releases.
Yeah, I wanted to post a general reminder in addition to the other issues. I noticed one collection which tagged releases inconsistently that's not in the seven above, and I didn't check for collections that may violate the guideline we added about reproducibility1. I think a general reminder would be useful. Footnotes
|
|
How's this? Hi! The Ansible Community Steering Committee has determined that this collection does not tag its releases in its git repository. This violates the repository management section of the Collection Requirements:
Note that this requirement has recently been clarified, but its intent remains the same. Please tag at least the previous 1-2 releases of your collection to come into compliance. Please keep us updated and let us know if you have any questions. Thanks! |
Personally, I'd remove the "too much". Apart from this your suggestion looks good to me. |
|
Wordsmithing a bit: "... clarified, but its intent remained the same." BTW, @mariolenz , @gotmax23 , @felixfontein thanks for chasing these issues. |
|
I've reported issues against the offending collections. Please the edits I've made to the issue description. I skipped the two collections that are already slated for removal. |
I've reported an issue against |
|
@mariolenz good point. I guess what happens with the issue there also helps us deciding whether it is still maintained :) |
This adds a new validate-tags command to retrieve the tags in each collection's git repository and ensure that the current version is tagged. Currently, this is a separate command, but I'd like to have `antsibull-build` include the data from `get_collections_tags()` in ansible-build-data and the ansible sdist after this has gotten more testing/feedback. I also split CollectionsMetadata into a separate file and added the new keys. Previously, this class was part of changelog.py and only used for retrieving changelog URLs. Relates: ansible-community/community-topics#148 Depends-on: ansible-community/ansible-build-data#176
|
I've submitted ansible-community/ansible-build-data#176 to add a data file with collection's repository URLs. I also submitted ansible-community/antsibull-build#456 to add a |
This adds a new validate-tags command to retrieve the tags in each collection's git repository and ensure that the current version is tagged. Currently, this is a separate command, but I'd like to have `antsibull-build` include the data from `get_collections_tags()` in ansible-build-data and the ansible sdist after this has gotten more testing/feedback. I also split CollectionsMetadata into a separate file and added the new keys. Previously, this class was part of changelog.py and only used for retrieving changelog URLs. Relates: ansible-community/community-topics#148 Depends-on: ansible-community/ansible-build-data#176
This adds a new validate-tags command to retrieve the tags in each collection's git repository and ensure that the current version is tagged. Currently, this is a separate command, but I'd like to have `antsibull-build` include the data from `get_collections_tags()` in ansible-build-data and the ansible sdist after this has gotten more testing/feedback. I also split CollectionsMetadata into a separate file and added the new keys. Previously, this class was part of changelog.py and only used for retrieving changelog URLs. Relates: ansible-community/community-topics#148 Depends-on: ansible-community/ansible-build-data#176
This adds a new validate-tags command to retrieve the tags in each collection's git repository and ensure that the current version is tagged. Currently, this is a separate command, but I'd like to have `antsibull-build` include the data from `get_collections_tags()` in ansible-build-data and the ansible sdist after this has gotten more testing/feedback. I also split CollectionsMetadata into a separate file and added the new keys. Previously, this class was part of changelog.py and only used for retrieving changelog URLs. Relates: ansible-community/community-topics#148 Depends-on: ansible-community/ansible-build-data#176
* Add validate-tags command This adds a new validate-tags command to retrieve the tags in each collection's git repository and ensure that the current version is tagged. Currently, this is a separate command, but I'd like to have `antsibull-build` include the data from `get_collections_tags()` in ansible-build-data and the ansible sdist after this has gotten more testing/feedback. I also split CollectionsMetadata into a separate file and added the new keys. Previously, this class was part of changelog.py and only used for retrieving changelog URLs. Relates: ansible-community/community-topics#148 Depends-on: ansible-community/ansible-build-data#176 * Fix validation of validate-tags options We should ensure that the input file exists (if that option is passed) and that the deps file exists to avoid exceptions down the line. Also, this fixes a typo in _normalize_validate_tags_options. * Add 456-validate-tags.yaml changelog fragment * Remove duplicate attribute type defs * Define tag regex as a file-level constant * validate_tags: Return list of errors instead of int * Make get_collections_tags() a library function Pass needed variables as function arguments instead of using app_ctx. * Add validate-tags-file subcommand and docstrings Remove --input from the validate-tags subcommand in favor of a separate validate-tags-file command. * get_tags: Set GIT_TERMINAL_PROMPT=0 We don't want git to ask for a password when a repository is inaccessible. The program should consider it a failure and move on. * validate-tags: Support a custom tag_version_regex Some collections don't tag versions as v{version} or {version}. While collections should use a standard tag name format per the Collection Requirements, we won't impose this on existing collections. * Fix type annotation and log collection name
|
I also opened issues against hpe.nimble and inspur.sm which tagged their other releases but not the most recent ones: |
|
We've followed up with cyberark.pas (no response and no action) and fortinet.fortimanager (response but no action). We plan to start the removal process for cyberark.pas if there's no response by next week. |
felixfontein
changed the title
|
@Andersson007 @felixfontein @russoz I think we should close this issue. Feel free to re-open it again if you think I'm wrong! |
|
Yes, let's close this. Ansible 10 has no more failures, and we are dealing with new ones as they come up. |
Summary
In light of the recent discussions about incongruities between galaxy versions and git tags, I programmatically went through all of the collections in the Ansible package, extracted their repository URL from the Galaxy API, and then got a list of their tags from Github.
The first thing I noticed is there are multiple collections that don't tag their releases at all. The following repositories do not have any git tags. Some of them seem to have release commits (e.g. F5Networks/f5-ansible@3ab145b), but I don't think that's sufficient.
https://github.com/cyberark/ansible-security-automation-collection (cyberark.pas) -- Collection Requirements Violation - Repository Management cyberark/ansible-security-automation-collection#46The fortinet collections mark releases by creating a separate branch for each release. Even under the current rules, I don't think this counts as tagging.
The text was updated successfully, but these errors were encountered: