From d55fb827252e2adaf0c4da57ab88b87f6b08a25e Mon Sep 17 00:00:00 2001 From: Felix Fontein Date: Sun, 1 Dec 2024 14:00:33 +0100 Subject: [PATCH] Document (re-)deprecation, cancelling deprecation, and re-adding collections; improve new major release docs; remove superfluous freeze docs (#502) * Document (re-)deprecation, cancelling deprecation, and re-adding collections. * Remove manual freeze instructions. Use the automated build process instead, which handles this already. * Update instructions for new major releases. * Remove removal block from example. * Add missing word. --- docs/automated-process.md | 2 +- docs/new-ansible-and-freezes.md | 165 ----------------- docs/new-ansible.md | 116 ++++++++++++ docs/policies.md | 311 +++++++++++++++++++++++++++++++- mkdocs.yml | 2 +- 5 files changed, 428 insertions(+), 168 deletions(-) delete mode 100644 docs/new-ansible-and-freezes.md create mode 100644 docs/new-ansible.md diff --git a/docs/automated-process.md b/docs/automated-process.md index 266ada4c8c..f46c7bfbbd 100644 --- a/docs/automated-process.md +++ b/docs/automated-process.md @@ -155,4 +155,4 @@ next to the release version: When the new major release has been done, remember to prepare the directory for the next major Ansible release. How to do this is described in [Setting up for a -new major release](new-ansible-and-freezes.md#setting-up-for-a-new-major-release). +new major release](new-ansible.md#setting-up-for-a-new-major-release). diff --git a/docs/new-ansible-and-freezes.md b/docs/new-ansible-and-freezes.md deleted file mode 100644 index a50dca7f40..0000000000 --- a/docs/new-ansible-and-freezes.md +++ /dev/null @@ -1,165 +0,0 @@ -# New Ansible Releases and Freezes - -## Preamble - -Releasing new Ansible major versions and frozen releases requires some special -handling. -For information about the general release process, -see [Ansible Release Process](release-process.md). - -## Setting up for a new major release - - - -After the release of `X.0.0`, it is necessary to create the directory -structure for Ansible `X+1`. - -1. Determine the current major version and next major version - - ``` sh - CURRENT_MAJOR_VERSION=9 - NEXT_MAJOR_VERSION=10 - ``` - -2. Create the major version directory - - ``` sh - mkdir "${NEXT_MAJOR_VERSION}/" - ``` - -3. Copy over the `ansible.in` and `collection-meta.yaml` files - - ``` sh - cp "${CURRENT_MAJOR_VERSION}/ansible.in" "${CURRENT_MAJOR_VERSION}/collection-meta.yaml" \ - "${NEXT_MAJOR_VERSION}/" - ``` - -4. Symlink `${CURRENT_MAJOR_VERSION}.0.0`'s deps file to - `${NEXT_MAJOR_VERSION}/ancestor.deps` - - ``` sh - ln -sr "${CURRENT_MAJOR_VERSION}/ansible-${CURRENT_MAJOR_VERSION}.0.0.deps" \ - "${NEXT_MAJOR_VERSION}/ancestor.deps" - ``` - -5. Create a stub `changelog.yaml` file - - ``` sh - cat <${NEXT_MAJOR_VERSION}/changelog.yaml - --- - ancestor: ${CURRENT_MAJOR_VERSION}.0.0 - releases: {} - EOF - ``` - -6. Create a blank `validate-tags-ignores` file - - ``` sh - touch "${NEXT_MAJOR_VERSION}/validate-tags-ignores" - ``` - -7. Create a blank `ansible-${NEXT_MAJOR_VERSION}.constraints` file - - ``` sh - touch "${NEXT_MAJOR_VERSION}/ansible-${NEXT_MAJOR_VERSION}.constraints" - ``` - - You might need to fill this with some initial data. - -8. Add the next major version to ansible-build-data's CI - - Open `.github/workflows/antsibull-build.yml` and the following block to the - matrix: - - ``` yaml - - name: Ansible ${NEXT_MAJOR_VERSION} - ansible_version: ${NEXT_MAJOR_VERSION}.99.0 - ansible_major_version: ${NEXT_MAJOR_VERSION} - ``` - -9. Commit the changes - - ``` sh - git add "${NEXT_MAJOR_VERSION}" .github/workflows/antsibull-build.yml - ``` - -10. Submit a PR against ansible-build-data - - -## Freeze release - - - -Beyond the regular [Ansible Release Process](release-process.md), X.Y.0 -releases require special handling before running the release playbook. - -!!! note - This can be done in a simpler fashion when using the [automated release - process](automated-process.md). See the regular automated workflow for - versions up to rc1, and the [Special builds](automated-process.md#special-builds) - section for further release candidates and for the final major release. - -1. Determine the previous and the current releases - - ``` sh - MAJOR_VERSION=9 - VERSION=9.0.0 - PREVIOUS_VERSION=9.0.0rc1 - ``` - -2. Set up your Git clones and release venv as outlined in the [Ansible Release Process](release-process.md) document. - -3. Copy over the previous release's deps and galaxy files. - - ``` sh - cp "ansible-${PREVIOUS_VERSION}.yaml" "ansible-${VERSION}.yaml" - cp "ansible-${PREVIOUS_VERSION}.deps" "ansible-${VERSION}.deps" - ``` - -4. Edit the current ansible version in the deps file - - ``` sh - sed -i "s|^_ansible_version:.*$|_ansible_version: ${VERSION}|" \ - "ansible-${VERSION}.deps" - ``` - -5. Add a changelog entry for the new release - - Open `${MAJOR_VERSION}/changelog.yaml` and the following block to the - releases table: - - ``` yaml - ${VERSION}: - changes: - release_summary: 'Release Date: ${RELEASE_DATE} - - - `Porting Guide `_' - release_date: '${RELEASE_DATE}' - ``` - - The release date should be formatted as `YYYY-MM-DD`. - -6. Manually update specific collection versions if needed. - - In some cases, it may be necessary to update certain collections if, for - example, a serious bug is found is one of the collections. - In that case, open up the deps and galaxy files copied over in the step 2 - and change the versions for the collections in question. - Make sure that the versions in both files are consistent. - -7. Generate the tags data file - - ``` sh - antsibull-build validate-tags \ - --data-dir . \ - --ignores-file validate-tags-ignores \ - --output "ansible-${VERSION}-tags.yaml" \ - "${VERSION}" - ``` - -8. Run the the release playbook as outlined in [Ansible Release Process](release-process.md). - Make sure to use pass `-e antsibull_data_reset=false` to preserve the - ansible-build-data modifications. diff --git a/docs/new-ansible.md b/docs/new-ansible.md new file mode 100644 index 0000000000..5ab105ef31 --- /dev/null +++ b/docs/new-ansible.md @@ -0,0 +1,116 @@ +# New Ansible Releases + +## Preamble + +Releasing new Ansible major versions and frozen releases requires some special +handling. +For information about the general release process, +see [Ansible Release Process](release-process.md). + +## Setting up for a new major release + + + +After the release of `X.0.0`, it is necessary to create the directory +structure for Ansible `X+1`. + +1. Determine the current major version and next major version: + + ``` sh + CURRENT_MAJOR_VERSION=11 + NEXT_MAJOR_VERSION=12 + ``` + +2. Create the major version directory: + + ``` sh + mkdir "${NEXT_MAJOR_VERSION}/" + ``` + +3. Copy over the `ansible.in` and `collection-meta.yaml` files: + + ``` sh + cp "${CURRENT_MAJOR_VERSION}/ansible.in" "${CURRENT_MAJOR_VERSION}/collection-meta.yaml" \ + "${NEXT_MAJOR_VERSION}/" + ``` + +4. Symlink `${CURRENT_MAJOR_VERSION}.0.0`'s deps file to + `${NEXT_MAJOR_VERSION}/ancestor.deps`: + + ``` sh + ln -sr "${CURRENT_MAJOR_VERSION}/ansible-${CURRENT_MAJOR_VERSION}.0.0.deps" \ + "${NEXT_MAJOR_VERSION}/ancestor.deps" + ``` + +5. Create a stub `changelog.yaml` file: + + ``` sh + cat <${NEXT_MAJOR_VERSION}/changelog.yaml + --- + ancestor: ${CURRENT_MAJOR_VERSION}.0.0 + releases: {} + EOF + ``` + +6. Create a blank `validate-tags-ignores` file: + + ``` sh + touch "${NEXT_MAJOR_VERSION}/validate-tags-ignores" + ``` + +7. Create a blank `ansible-${NEXT_MAJOR_VERSION}.constraints` file: + + ``` sh + touch "${NEXT_MAJOR_VERSION}/ansible-${NEXT_MAJOR_VERSION}.constraints" + ``` + + You might need to fill this with some initial data. + +8. Add the next major version to ansible-build-data's CI: + + Open `.github/workflows/antsibull-build.yml` and add the following block to the + matrix: + + ``` yaml + - name: Ansible ${NEXT_MAJOR_VERSION} + ansible_version: ${NEXT_MAJOR_VERSION}.99.0 + ansible_major_version: ${NEXT_MAJOR_VERSION} + ``` + +9. Update `collection-meta.yaml` and `ansible.in`: + + 1. Find all collection entries from `collections` in `collections-meta.yaml` + that have a `removal` entry with `major_version` equal to `${NEXT_MAJOR_VERSION}`. + + - If the `removal` entry has an `updates` and the last entry is about keeping them (`readded_version` or `cancelled_version`), + remove the `removal` entry completely. + + - Otherwise, cut out the collection's entry and paste it into a temporary buffer / file. + + 2. For every cut out metadata entry, + remove the corresponding entry in `ansible.in`. + + 3. For every cut out metadata entry, + find the correct place in `removed_collections` in `collections-meta.yaml` + to insert the removed part from `collections-meta.yaml` and paste the entry there. + Modify the entry as follows: + + - Replace `major_version: ${NEXT_MAJOR_VERSION}` by `version: ${NEXT_MAJOR_VERSION}.0.0a1`. + + - Remove `announce_version` if present. + + - Remove `updates` if present. + +10. Validate build data: + + ```sh + antsibull-build lint-build-data --data-dir "${NEXT_MAJOR_VERSION}" "${NEXT_MAJOR_VERSION}" + ``` + +11. Commit the changes: + + ``` sh + git add "${NEXT_MAJOR_VERSION}" .github/workflows/antsibull-build.yml + ``` + +12. Submit a PR against [the ansible-build-data repository](https://github.com/ansible-community/ansible-build-data/). diff --git a/docs/policies.md b/docs/policies.md index ac4f5883e1..3e770c2856 100644 --- a/docs/policies.md +++ b/docs/policies.md @@ -6,7 +6,316 @@ and [Repository management][2]. ## Removal from Ansible -TODO +### Announce removal of a collection (deprecation) + +If a collection should be removed in a future Ansible version, +its removal should be announced in all current major releases, +and should also be announced in the upcoming major release +(unless it is removed from that version - [see the next subsection for that](#removing-a-collection)). + +To announce removal, removal metadata needs to be added to the collection metadata in `collection-meta.yaml`. +Assume you have the following collection metadata: +```yaml +collections: + foo.bar: + maintainers: + - Foo Bar + repository: https://github.com/ansible-collections/foo.bar +``` +Then a `removal` subkey needs to be added to `foo.bar` with the following fields: + +- `major_version`: the major Ansible version from which the collection shall be removed. + +- `reason`: can be one of: + + 1. `deprecated`: the collection has been deprecated by its maintainers. + ([Official process]( + https://docs.ansible.com/ansible/devel/community/collection_contributors/collection_package_removal.html#removing-a-collection-that-has-been-explicitly-deprecated-or-abandoned-by-its-former-maintainers).) + + 2. `considered-unmaintained`: the collection is considered unmaintained by the Steering Committee. + ([Official process]( + https://docs.ansible.com/ansible/devel/community/collection_contributors/collection_package_removal.html#identifying-and-removing-an-unmaintained-collection-that-has-not-been-deprecated-by-its-maintainers).) + + 3. `renamed`: the collection has been renamed. + The new name of the collection should be specified in `new_name`. + Also `redirect_replacement_major_version` should be added + with the major Ansible release that will contain only deprecated redirects to the new collections. + + Note that in this case, `major_version` can have the special value `TBD` + for when it is not clear when the old collection will eventually be removed from Ansible yet. + + ([Official process]( + https://github.com/ansible-community/ansible-build-data/?tab=readme-ov-file#renaming-a-collection).) + + 4. `guidelines-violation`: the collection has been removed by the Steering Committee due to guidelines violation. + Further details must be provided in `removal_text`. + ([Official process]( + https://docs.ansible.com/ansible/devel/community/collection_contributors/collection_package_removal.html#collections-not-satisfying-the-collection-requirements).) + + 5. `other`: the collection has been removed for other reasons. + Further explanation on why the collection will be removed must be provided in `removal_text`. + +- `announce_version`: optional string to indicate in which release of this Ansible major version the removal should be announced. + When adding a new removal, use the next version that will be used for a release. + A corresponding changelog entry will automatically be added to this version in the Ansible changelog. + +- `discussion`: optional string with an URL to the removal discussion in the forum. + This link will be mentioned in the generated changelog entries and on the docsite. + +The following are a few examples of how `removal` metadata can look: +```yaml +collections: + foo.bar_deprecated: + # In case a collection has been deprecated/abandoned by its maintainers. + repository: https://github.com/ansible-collections/foo.bar + removal: + major_version: 20 + reason: deprecated + announce_version: 11.2.0 + discussion: https://forum.ansible.com/t/.../ + foo.bar_unmaintained: + # In case the Steering Committee decided that a collection is effectively unmaintained. + repository: https://github.com/ansible-collections/foo.bar + removal: + major_version: 20 + reason: considered-unmaintained + announce_version: 11.2.0 + discussion: https://forum.ansible.com/t/.../ + foo.bar_renamed: + # Use this in case a collection has been renamed. + repository: https://github.com/ansible-collections/foo.bar + removal: + major_version: 20 # can be TBD if not yet known + reason: renamed + new_name: foo.bar_new_name + redirect_replacement_major_version: 13 # leave away if not yet known + announce_version: 11.2.0 + discussion: https://forum.ansible.com/t/.../ + foo.bar_guidelines_violation: + # In case the Steering Committe decided to remove the collection for guidelines violation. + repository: https://github.com/ansible-collections/foo.bar + removal: + major_version: 20 + reason: guidelines-violation + removal_text: >- + Extra text that must specify what happened. Can use L(Ansible markup, + https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_documenting.html#linking-within-module-documentation). + announce_version: 11.2.0 + discussion: https://forum.ansible.com/t/.../ + foo.bar_other: + # If the Steering Committee decides to remove a collection for a non-predefined reason. + repository: https://github.com/ansible-collections/foo.bar + removal: + major_version: 20 + reason: other + removal_text: >- + Text that must specify why the collection was removed. Can use L(Ansible markup, + https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_documenting.html#linking-within-module-documentation). + announce_version: 11.2.0 + discussion: https://forum.ansible.com/t/.../ +``` + +You can use `antsibull-build lint-build-data --data-dir /path/to/ansible-build-data/X/ X` +(where `X` is the major version) +to validate the entries. + +### Removing a collection + +If a collection should be removed from the upcoming Ansible major version, +for which a metadata directory already exists but for which feature freeze has not yet happened, +the collection needs to be removed from the build metadata. +Also note that the removal should be announced in existing major versions, +[see the previous section for details on that](#announce-removal-of-a-collection-deprecation). + +First, remove the collection's entries from `ansible.in`, `ansible-X.build`, and if necessary, `ansible-X.constraints`. + +Then the metadata in `collections-meta.yaml` needs to be updated: + +1. For that, locate the collection in the `collections` list. + Select the entry, copy it to the clipboard, and remove it from `collections`. + +2. Locate the `removed_collections` list and the position where the collection should be inserted. + Paste the copied entry from the clipboard. + +3. If `removal` already exists, edit it as follows: + + - Replace `major_version` by `version`, + and put in the exact Ansible version from which the collection is removed. + **Note that this should not happen after the feature freeze!** + + - If `announce_version` is present, remove it. + This removes the corresponding changelog entry if the Ansible changelog is regenerated, + but that entry should not have been there + unless the major version for removal got changed + (in that case add an explicit manual entry to `changelog.yaml` if you want to keep the old entry). + +4. If `removal` does not yet exist, + create it [as described in the previous section](#announce-removal-of-a-collection-deprecation), + with the changes as described in the previous point. + +As an example, consider the following metadata entry: +```yaml +collections: + foo.bar: + repository: https://github.com/ansible-collections/foo.bar +``` +This should be moved to `removed_collections` and changed as follows: +```yaml +removed_collections: + foo.bar: + repository: https://github.com/ansible-collections/foo.bar + removal: + version: 20.0.0a1 + reason: deprecated + discussion: https://forum.ansible.com/t/.../ +``` + +You can use `antsibull-build lint-build-data --data-dir /path/to/ansible-build-data/X/ X` +(where `X` is the major version) +to validate the entries. + +### Cancel deprecation of a collection + +If a collection that is scheduled for removal in an upcoming major release should be kept, +the removal metadata needs to be adjusted. +The collection needs to be re-added in the build metadata for the upcoming major release +if it has already been removed from there; +[details for this can be found in the next subsection](#re-adding-a-already-removed-collection). + +Locate the `removal` entry for the collection in `collection-meta.yaml`'s `collections` list, +and update it as follows: + +1. If not there, add a `updates:` subkey. + +2. Add an entry with `cancelled_version` (the Ansible release where the cancelation should be announced in the changelog) + and `reason_text` (explanation), and optionally `discussion` (if a different discussion URL should be used). + +This can look as follows: +```yaml +collections: + foo.bar: + repository: https://github.com/ansible-collections/foo.bar + removal: + major_version: 20 + reason: deprecated + announce_version: 12.3.0 + discussion: https://forum.ansible.com/t/.../ + updates: + - cancelled_version: 12.5.0 + reason_text: Maintenance of the collection has been taken over by L(someone else, https://...). +``` + +You can use `antsibull-build lint-build-data --data-dir /path/to/ansible-build-data/X/ X` +(where `X` is the major version) +to validate the entries. + +### Re-adding a already removed collection + +If a collection that has already been removed from the upcoming major version should be kept, +the removal metadata needs to be adjusted. +The removal needs to be canceled for previous major releases, +[details for this can be found in the prevous subsection](#cancel-deprecation-of-a-collection). + +First, add the collection's entries to `ansible.in` and `ansible-X.build`. + +Then the metadata in `collections-meta.yaml` needs to be updated: + +1. For that, locate the collection in the `removed_collections` list. + Select the entry, copy it to the clipboard, and remove it from `removed_collections`. + +2. Locate the `collections` list and the position where the collection should be inserted. + Paste the copied entry from the clipboard. + +3. If not there, add a `updates:` subkey. + +4. Remove `announce_version` from the main `removal` entry and add it to a new `updates` entry with key `removed_version`. + +5. Add an `updates` entry with `readded_version` (the Ansible release where the re-add should be announced in the changelog) + and `reason_text` (explanation), and optionally `discussion` (if a different discussion URL should be used). + +As an example, consider the following metadata entry: +```yaml +removed_collections: + foo.bar: + maintainers: + - Foo bar + repository: https://github.com/ansible-collections/foo.bar + removal: + version: 11.0.0a1 + reason: considered-unmaintained + discussion: https://forum.ansible.com/t/.../ + announce_version: 10.0.0a1 +``` +This should be moved to `collections` and changed as follows: +```yaml +collections: + foo.bar: + maintainers: + - Foo bar + repository: https://github.com/ansible-collections/foo.bar + removal: + major_version: 11 + reason: considered-unmaintained + discussion: https://forum.ansible.com/t/.../ + updates: + - removed_version: 11.0.0a1 + - readded_version: 11.0.0b2 + reason_text: Maintenance of the collection has been taken over by L(another team, https://...). +``` + +You can use `antsibull-build lint-build-data --data-dir /path/to/ansible-build-data/X/ X` +(where `X` is the major version) +to validate the entries. + +### Re-deprecating a collection + +It can happen that a collection is scheduled for removal (deprecated) +that already was scheduled for removal before (or even has been removed before), +but the removal was canceled or the removed collection re-added. +In that case, updating the build metadata is a bit different +from [announcing its removal the first time](#announce-removal-of-a-collection-deprecation). + +Then the metadata in `collections-meta.yaml` needs to be updated. +The collection should be in `collections` and already have a `removal` subkey, +and that should have a `updates` subkey. + +Simply add a new entry there with: + +1. `redeprecated_version`: the exact Ansible version when the re-deprecation should be announced. + +2. `discussion`: optional URL if a new discussion thread is used on the forum. + If not provided, `discussion` from the `removal` entry will be used. + +3. `reason`: a reason why the collection has been re-deprecated. + Can be one of `deprecated`, `considered-unmaintained`, `renamed`, `guidelines-violation`, or `other`. + +4. `reason_text`: can only be provided if `reason` is not provided, or if it is one of `other` and `guidelines-violation`. + It always must be provided if `reason` is one of `other` and `guidelines-violation`. + +For example: +``` +collections: + foo.bar: + maintainers: + - Foo bar + repository: https://github.com/ansible-collections/foo.bar + removal: + major_version: 11 + reason: considered-unmaintained + discussion: https://forum.ansible.com/t/.../ + updates: + - removed_version: 11.0.0a1 + - readded_version: 11.0.0b2 + reason_text: Maintenance of the collection has been taken over by L(another team, https://...). + - redeprecated_version: 11.3.0 + discussion: https://... + reason: guidelines-violation + reason_text: The guideline L(XXX, https://...) was violated. +``` + +You can use `antsibull-build lint-build-data --data-dir /path/to/ansible-build-data/X/ X` +(where `X` is the major version) +to validate the entries. ## Repository management diff --git a/mkdocs.yml b/mkdocs.yml index 3f99966300..eb874ead51 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -30,4 +30,4 @@ nav: - Automated Ansible Release Process: automated-process.md - Manual Ansible Release Process: release-process.md - policies.md - - new-ansible-and-freezes.md + - new-ansible.md