HOWTO-RELEASE

	Notes on Preparing a GDAL Source Release
	========================================


Prerequisites:

1) Check that the release is ready to go as far as ABI (binary compatibility)
   is concerned. This can be checked by comparing the installed headers of the
   candidate release with the installed headers of the previous release
   (diff -ur $(OLD_INSTALL_DIR)/include $(NEW_INSTALL_DIR)/include). The API
   is defined as all functions and classes exported by the CPL_DLL keyword.

  - For major and minor releases, there must be no function signature change
    for the C API. Only new functions are allowed.

  - For major releases, the allowed changes in C++ API should (or must?) be
    such that user calling C++ code can still compile against new headers
    without modification (existing methods can become virtual, default
    arguments can be added, new methods or members can be added)

  - For minor releases (1.6.1 versus 1.6.0), the C++ ABI stability must be
    preserved : no method signature change, no addition of virtual methods, no
    new members. Only non-virtual methods can be added.

    It may also be helpful to check:
      https://abi-laboratory.pro/tracker/timeline/gdal/

2) Delete existing fix_typos directory and run scripts/fix_typos.sh

Process :

1) a) Regenerate configure using autogen.sh and commit if changed.
   b) Regenerate swig generated files for python bindings and commit if changed.

   There is often a reference system on which this should be done (i.e. Frank's
   dev workstation) to avoid unnecessary churn from different autoconf or swig
   versions.

   c) "cd scripts; make completion" to regenerate scripts/gdal-bash-completion.sh
      if new command line switches have been added. scripts/completionFinder.py
      must also be edited before if new utilities/scripts are added/removed.

2.1) Update the release date, and number information in gcore/gdal_version.h.in
     (*NOT* gdal_version.h which is a generated file)
     Note: the format of GDAL_RELEASE_DATE should be YYYYMMDD.

2.2) Update two instances of year in ../CITATION file to the current year.

3) Update the VERSION file (if not already done)

3.1) Update the version information in the following files:
     - ./swig/python/setup.py (gdal python package)
     - ./swig/python/gdal-utils/osgeo_utils/__init__.py (gdal-utils python package)
     - ./swig/python/README.rst (libgdal)

3.2) For major releases update the VERSION macro in nmake.opt (for 1.6, 1.7etc)

4) Update LIBGDAL_CURRENT/REVISION/AGE macros in GDALmake.opt.in.
   - For a release with no interface changes just bump REVISION.
   - Adding interfaces, bump CURRENT/AGE, set REVISION to 0.
   - Deleting/changing interfaces in C or C++ API, or modification of
     the C++ ABI (e.g. addition of virtual method, new argument with default value, etc.):
     bump CURRENT, set others to zero.

5) Prepare release overview in the NEWS file. For example, to get all changes
   from v3.4.0 to current HEAD
     git log --reverse -v v3.4.0..HEAD . ":(exclude)autotest" ":(exclude)doc" ":(exclude).github"

  - commit new version to NEWS file.

  Create a News page in Trac for the release, like
    http://trac.osgeo.org/gdal/wiki/Release/3.1.0-News

6) If this is a feature release (e.g 3.1), prepare a branch.

   git checkout master
   git pull origin master
   git checkout -b release/3.1
   git push origin release/3.1

7) Tag the release with a RC suffix:

   git checkout release/3.1
   git pull origin release/3.1
   git tag -a -m "Create tag v3.1.0RC1" v3.1.0RC1
   git push origin v3.1.0RC1

8) Prepare archives

8.0) Ensure you have the following prerequisites (beyond normal build deps):

   If make is not GNU make, e.g., export MAKE=gmake

   doxygen

   python available as "python", or e.g. export PYTHON=python3.9

   python3 available as "python3" in path (export PYTHON=python3.9
   partially works, but some scripts under doc have hardcoded python3)

   sphinx-build in path, or e.g. export SPHINXBUILD=sphinx-build-3.9

   breathe extension for sphinx

   swig 4 in path as "swig"

   md5sum (GNU version) in path (not a POSIX requirement)

8.1) Create the source distributions using the mkgdaldist.sh script.
   The argument should be the version number (i.e. 1.4.2). As our process involves
   doing betas or RCs, use the -rc option so that the filenames include this
   information (after promotion to official release, filename renaming will have
   to be done)

   % ./mkgdaldist.sh 3.1.0 -tag v3.1.0RC1 -rc rc1

   For a beta version:

   % ./mkgdaldist.sh 3.1.0beta1 -tag v3.1.0beta1

   For a version intended for testing packaging (and not for
   distribution):

   % ./mkgdaldist.sh 3.4.80 -tag master

   It might be useful to compare the content of the generated tarball, with
   a previous release (diff -Nur gdal-3.0.1 gdal-3.0.2).
   This is more easily doable for a bugfix release.

8.2) Create a snapshot of the documentation (only for feature releases)

    0. Potentially refresh
       https://download.osgeo.org/gdal/for_doc/javadoc.zip (./make_doc.sh from swig/java)
       https://download.osgeo.org/gdal/for_doc/python-doc.tar.gz (make doc-tarball from swig/python)
    1. cd doc
    2. ./build_doc_snapshot 310
       This generates gdal310doc.zip

8.3) Publish the resulting files on download.osgeo.org,
    in /osgeo/download/gdal/X.Y.Z (where X.Y.Z is the version number)

    with ~/.ssh/config containing:

   Host osgeo7-*
        ProxyCommand ssh rouault@download.osgeo.org -W $(sed -e "s/^osgeo7-//;s/$/.lxd/" <<< "%h"):%p


9) Announce the release candidate availability to gdal-dev@
   Example: https://lists.osgeo.org/pipermail/gdal-dev/2019-June/050509.html

10) If new RC is needed, update gcore/gdal_version.h.in with the new date
    and update NEWS,and go to 7)

    If no new RC is needed, after a few days, raise a motion to gdal-dev@ for approval

11) Once the vote has positively completed,

11.1) Tag the release as final:

    git checkout v3.1.0RC1
    git tag -a -m "Create tag v3.1.0" v3.1.0
    git push origin v3.1.0

11.2) Log on download.osgeo.org, go to /osgeo/download/gdal/X.Y.Z

      Remove the RC suffixes for the final RC, like:
      mv gdal-3.1.0rc1.tar.xz gdal-3.1.0.tar.xz
      mv gdal-3.1.0rc1.tar.xz.md5 gdal-3.1.0.tar.xz.md5
      mv gdal-3.1.0rc1.tar.gz gdal-3.1.0.tar.gz
      mv gdal-3.1.0rc1.tar.gz.md5 gdal-3.1.0.tar.gz.md5
      mv gdal310rc1.zip gdal310.zip
      mv gdal310rc1.zip.md5 gdal310.zip.md5
      mv gdalautotest-3.1.0rc1.tar.gz gdalautotest-3.1.0.tar.gz
      mv gdalautotest-3.1.0rc1.zip gdalautotest-3.1.0.zip

      And edit the .md5 file to remove the reference to RC in the names they contain
      vim gdal-3.1.0.tar.gz.md5
      vim gdal-3.1.0.tar.xz.md5
      vim gdal310.zip.md5

      Check that everything is fine:
      md5sum -c *.md5

11.3) In /osgeo/download/gdal, add a symlink from X.Y.Z to CURRENT (except for stable releases in a "old" branch).

  % ln -sf X.Y.Z CURRENT

12) (Removed)

13) Update doc/source/about_no_title.rst and doc/source/download.rst to advertise the new release and link to the release notes

14) Update GitHub to close the release milestone.
    Then create a new milestone for the next release.

15) Upload the new Python bindings to Pypi (requires upload rights to
the GDAL package by one of the current owners : HowardB/FrankW/EvenR)
( procedure taken from http://peterdowns.com/posts/first-time-with-pypi.html )

15.1) Prerequisite:
    a) Install twine https://pypi.org/project/twine/

    b) Create a $HOME/.pypirc file :

    [distutils] # this tells distutils what package indexes you can push to
    index-servers = pypi
                    pypitest

    [pypi] # authentication details for live PyPI
    repository: https://upload.pypi.org/legacy/
    username: yourlogin
    password: yourpassword

    [pypitest] # authentication details for test PyPI
    repository: https://test.pypi.org/legacy/
    username: yourlogin
    password: yourpassword

15.2) create gdal sdist and upload to pypi:

    a) cd swig/python

    b) python setup.py sdist

    c) Check the output:
        twine check dist/GDAL-*.gz

    d) For trial :
        twine upload dist/GDAL-*.gz -r pypitest

    e) For real :
        twine upload dist/GDAL-*.gz

15.3) create gdal-utils wheel and upload to pypi:

    a) cd swig/python/gdal-utils

    b) python setup.py bdist_wheel

    c) Check the output:
        twine check dist/gdal_utils-*.whl

    c) For trial :
        twine upload dist/gdal_utils-*.whl -r pypitest

    d) For real :
        twine upload dist/gdal_utils-*.whl

16) Build and bundle the java bindings.

a) cd swig/java

b) Make any updates to java.opt that might be required for the platform. For
   example osx and windows users will have to change JAVA_INCLUDE to contain
   include/darwin and include/win32 respectively.

c) Build the bindings:

     make

17) Build maven artifacts. NOTE: This step only works on Linux and OSX since it
    requires a shell script to build javadocs.

    A quick one-liner for this step is:

      ant maven_sign -Dgpg.key=... -Dgpg.pass=...

    This will build and sign the artifacts with a single command. Read on for
    more details.

a)  Build the maven artifacts:

     ant maven

    Upon success maven artifacts should be located in the build/maven directory.

b)  Sign maven artifacts with GPG. This step is required in order to deploy the
    maven artifacts to the central Maven repository. Before this step can
    proceed you must set up a signing key as described here:

      http://central.sonatype.org/pages/working-with-pgp-signatures.html

    Each developer can use their own signing key although it is usually best to
    avoid constantly using a different key for releases as users will need to
    import the public key in order to verify the artifacts.

    Here are a quick set of steps to generate a signing key key.

       gpg --gen-key
       gpg --list-keys (and note the key id)
       gpg --keyserver hkp://pool.sks-keyservers.net --send-keys <key id>

    See the above link for more details.

    Once a key is set up run the "maven_sign" ant target. The target takes two
    parameters specified as system properties:

    1. gpg.key - The identifier for the signing key
    2. gpg.pass - The passphrase for the signing key (optional)

      ant maven_sign -Dgpg.key=... -Dgpg.pass=...

    Upon success you should see maven artifacts along with generated signatures
    in the build/maven directory. You will also find a file named "bundle.jar"
    that contains all the maven artifacts with signatures. This file is what
    will be uploaded to maven central. See the next step.

18) Deploy maven artifacts to Maven central.

    NOTE: Before you can deploy to maven central you must set up an account
    in Sonatype JIRA. That can be done here:

        https://issues.sonatype.org/secure/Signup!default.jspa

    Once you have an account set up you must be associated with the gdal
    project. Create a ticket here asking to be associated with the project:

        https://issues.sonatype.org/browse/OSSRH

    The entire deployment process is described in detail here:

       http://central.sonatype.org/pages/manual-staging-bundle-creation-and-deployment.html
       http://central.sonatype.org/pages/releasing-the-deployment.html

    The following steps summarize the process.

a) Log into the Sonatype repository manager at https://oss.sonatype.org. Use the
   same credentials as your Sonatype JIRA account.

b) Once log in select "Staging Upload" on the left hand side.

c) Select "Artifact Bundle" under "Upload Mode" and then choose the "bundle.jar"
   created in the previous Step 22. Finally "Upload Bundle" to start the upload.

d) When the upload has been completed you will be notified that a staging
   repository has been created. Note the name of the repository. It should look
   something like "orggdal-100x".

e) From the left hand menu navigate to "Staging Repositories". In the search
   box look for the staging repository name you noted from the previous section.
   Or just search for "gdal". It should be obvious which repository is the
   current one.

f) Select the staging repository. If all is well You should see the option to
   "Release" (Located as a button near the top of the page). If not it means
   there was an issue with the bundle. Consult the "Activity" tab at the bottom
   of the page to find out why.

e) Click the "Release" button and that is it! The release should be available in
   Maven Central shortly. You can verify this by going to search.maven.org and
   searching for "gdal".

19) Create a release on GitHub
    Go to https://github.com/OSGeo/gdal/releases/new
    And create a release for the vX.Y.Z tag
    Attach the gdal-X.Y.Z.tar.gz, gdal-X.Y.Z.tar.gz.md5, gdalXYZ.zip, gdalXYZ.zip.md5 and gdalautotest-X.Y.Z.zip files

20) Regenerate Docker images

    e.g
        ./docker/build-all.sh --with-multi-arch --release --tag 3.3.1 --gdal v3.3.1 --proj 8.1.0 --platform linux/arm64,linux/amd64 --push

    and update docker/README.md with the latest release

21) Announce release to :
    - major release: gdal-dev@lists.osgeo.org, gdal-announce@lists.osgeo.org, news_item@osgeo.org.
    - bugfix release: gdal-dev@lists.osgeo.org, gdal-announce@lists.osgeo.org

   Example:
    - bugfix release: https://lists.osgeo.org/pipermail/gdal-dev/2019-July/050543.html
    - feature release: https://lists.osgeo.org/pipermail/gdal-dev/2020-May/052117.html

Note: gdal-announce@ is moderated. Make sure that your email address is approved
('mod' tick disabled in https://lists.osgeo.org/mailman/admin/gdal-announce/members),
or your message manually approved, with an administrator of the list.

22) For a feature release: after it has been done, in the master branch, update
    the VERSION to the next one and in gcore/gdal_version.h.in, update
    GDAL_VERSION_MAJOR/_MINOR, GDAL_RELEASE_NAME (with a dev suffix, like "3.5.0dev"),
    and set GDAL_RELEASE_DATE to a date like {YEAR}9999.