Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change release.py to use the new centralized tarball approach #1048

Merged
merged 17 commits into from
Nov 3, 2023

Conversation

j-rivero
Copy link
Contributor

@j-rivero j-rivero commented Oct 26, 2023

The PR changes the release.py logic to be able to use the new generation of tarball that happens on Jenkins instead of using the dev local machines. See the linked docs update for a general usage and data flow overview gazebosim/docs#403

No changes in releasing workflow and the release.py call: Note that the tag of the project code to be built still happens in the local dev system so the workflow is the same than now: go to the source code you want to build and run release.py with the same arguments than now. Instead of calling -debbuilders there will a chain of actions composed

PR is quite long, probably easier to read change by change to understand the logic:

  1. Remove all current code related to tarball generation 5f18737
  2. Remove current --no-generate-source-file and clean up nightly generation 7887be0
  3. Helper function to get the github repository uri from current directory:
    1. Originally implemented in 6a712ff but patched afterwards in a870d8c#diff-9f016d3777d731463c2969cd7d16bb014363bee6bf98af9fe7ca66c4392949ad
    2. Sanity check for right format added in dc06b97
  4. Introduce new release.py parameters --source-repo-existing-ref and --source-tarball-uri. If --source-tarball-uri:
    1. Implement the transport of the parameters in the release call e786626
    2. Explain in --help what the new parameters do 8aca8ac
  5. Change logic to make the two modes of calling to work: db39135
    1. Standard new workflow calls to gz-foo-source directory as entry point
    2. Nightly and --source-tarball-uri calls will trigger gz-foo-debbuilders directly since no source code needs to be generated.
      1. Make the --bump-only-revision-linux parameter to need --source-tarball-uri cbb83e2

CI Testing:

  1. Implemented an smoke test job to be added to CI
    1. Change release.py to be use _RELEASEPY_TEST_RELEASE_REPO 4791392#diff-9f016d3777d731463c2969cd7d16bb014363bee6bf98af9fe7ca66c4392949ad
    2. See the bash testing in a870d8c#diff-9074386702f4ff2677975dcd91d17dd99887e1e2a886f8234056e4a8ebefbf9b

Manual testing:
Run the whole generation pipeline with the real example of gz-utils2 version 2.2.0~pre3. Steps were as follow:

  1. Usual previous steps to prepare the -release repository and the source code modifying CMakeLists.txt
  2. cd code/gz-utils
  3. Dry run ~/code/release-tools/release.py gz-utils2 2.2.0~pre1 <token> --dry-run --upload-to-repo prerelease to see the gz-source
  4. Run release ~/code/release-tools/release.py gz-utils2 2.2.0~pre1 <token> --upload-to-repo prerelease
    1. The release.py call triggers gz-utils2-source Build Status. Which after success, called:
    1. repository_uploader_packages to upload the tarball. In the parameters can be found the new tarball name and upload target. Build Status
    2. _releasepy (that will wait until all jobs of repository_uploader_packages finish) to call the debbuilders using --source-tarball-uri that points to the new tarball. The call and output is hidden in the a workspace log for security reasons. Build Status. This job will call --debuilders and the brew bottle builder as it is done until now.

Copy link
Contributor

@nuclearsandwich nuclearsandwich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than a minor typo all good!

check_releasepy.bash Outdated Show resolved Hide resolved
Co-authored-by: Steven! Ragnarök <[email protected]>
release.py Outdated Show resolved Hide resolved
Co-authored-by: Steve Peters <[email protected]>
@j-rivero j-rivero merged commit 9c7d545 into master Nov 3, 2023
1 check passed
@j-rivero j-rivero deleted the jrivero/new_releasepy_clean_rebase branch November 3, 2023 16:35
j-rivero added a commit that referenced this pull request Nov 7, 2023
* Remove all code related to tarball generation
* Implement new parameters and logic for calling -source or -debbuidlers when needed
* Implement a test script for release.py and use it in Github Actions.

---------

Signed-off-by: Jose Luis Rivero <[email protected]>
Co-authored-by: Steven! Ragnarök <[email protected]>
Co-authored-by: Steve Peters <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants