Add 3.1 porting guide

docs/index.rst

@@ -109,7 +109,7 @@ Collection maintainers can learn to correctly declare dependencies for their col
    definition
    usage
    collection_metadata
-   porting_guide_v3
+   porting_guides/porting_guide
    glossary
 
 .. toctree::

docs/porting_guides/porting_guide.rst

@@ -0,0 +1,13 @@
+******************************
+Ansible Builder Porting Guides
+******************************
+
+This section lists porting guides that can help you in updating your Execution Environment files between
+versions of ``ansible-builder``.
+
+
+.. toctree::
+   :maxdepth: 1
+
+   porting_guide_v3.1
+   porting_guide_v3.0

docs/porting_guide_v3.rst → docs/porting_guides/porting_guide_v3.0.rst

@@ -1,7 +1,8 @@
-.. _porting_guide_v3:
+.. _porting_guide_v3.0:
 
-Ansible Builder version 3 Porting Guide
-=======================================
+*********************************
+Ansible Builder 3.0 Porting Guide
+*********************************
 
 This section discusses the behavioral changes between ``ansible-builder`` version 1.2 and version 3.0.
 
@@ -110,4 +111,4 @@ version
 
 Must be ``3`` if you are using Ansible Builder 3.x.
 
-See the :ref:`version <version>` section for more details.
+See the :ref:`version <version>` section for more details.

docs/porting_guides/porting_guide_v3.1.rst

@@ -0,0 +1,120 @@
+*********************************
+Ansible Builder 3.1 Porting Guide
+*********************************
+
+This section discusses the behavioral changes between ``ansible-builder`` version 3.0 and version 3.1.
+
+.. note::
+
+    We highly advise running ``ansible-builder`` with increased verbosity (``--v3``) to fully expose any error
+    messages that may help in diagnosing any problems.
+
+.. contents:: Topics
+
+Python Requirements Handling
+============================
+
+The 3.1 release significantly changes how Python and system requirements are handled by simplifying dependency
+parsing. This release removes the use of an external library that was unmaintained and parsed many types of
+Python dependencies either partially or completely incorrectly. The changes are described below.
+
+PEP 508 Standard
+----------------
+
+Python requirements files are expected to follow the `PEP 508 standard <https://peps.python.org/pep-0508/>`_.
+Builder will *expect* the requirements file to be in this format, but it makes two exceptions:
+
+#. Comments (lines beginning with ``#``) are ignored.
+#. Any line from the requirements file that is not compliant with PEP508 causes a warning to be omitted, and
+   the line is passed through to ``pip`` unmodified. It is recommended to *not* depend on this behavior as we
+   cannot guarantee that these lines will not cause issues later during the dependency installation process.
+
+The passthrough of lines not following PEP508 may cause issues that may not have been present with version 3.0
+since many of these lines were simply ignored and not passed to ``pip`` with that version.
+
+Dependency Sanitization
+-----------------------
+
+Dependency sanitization (the combining of duplicate dependencies into a single dependency entry) is no longer performed.
+
+.. note::
+
+    The ``--sanitize`` option to the ``ansible-builder introspect`` command still exists, but is now undocumented
+    and does nothing.
+
+The effect of this change is that builder will now pass a listed dependency multiple times for each requirement file
+in which it is found. For example, with version 3.0, if collection A listed the Python dependency ``foo``, and
+collection B listed the dependency ``foo>=1.0``, then it would have appeared in the combined Python requirements file
+as a single entry:
+
+::
+
+    foo,foo>=1.0   # from collection A, B
+
+Now, with version 3.1, those dependencies are no longer combined and will appear in the combined Python requirements
+file as separate entries:
+
+::
+
+    foo  # from collection A
+    foo>=1.0  # from collection B
+
+If your container image has an older version of ``pip``, this change might cause an error during the image build
+process. This guide covers how to deal with this situation below.
+
+Common Issues
+=============
+
+This section lists some common errors that might be encountered when running ``ansible-builder`` version 3.1.
+
+ERROR: Double requirement given
+-------------------------------
+
+Because dependency sanitization has been removed, *all* Python requirements from included collections and user
+requirement files are passed along to the ``pip`` command that installs those requirements. Due to this change,
+the image build may exit abnormally with an error similar to the following:
+
+::
+
+    ERROR: Double requirement given: netaddr>=0.10.1 (from -r /tmp/src/requirements.txt (line 13)) (already in netaddr (from -r /tmp/src/requirements.txt (line 4)), name='netaddr')
+    Error: building at STEP "RUN /output/scripts/assemble": while running runtime: exit status 1
+
+This error comes from ``pip`` within one of the intermediate container images when attempting to install the Python
+requirements from included collections and/or from the user supplied requirements. This intermediate image is based
+on the base image defined within the Execution Environment file, and it means that the version of ``pip`` installed
+within that image is too old to handle duplicate requirement entries.
+
+To determine where the duplicate requirements are coming from, run ``ansible-builder`` with the ``--v3`` option
+to get more verbose output, then look for the output from the introspection phase. It will look similar to below:
+
+::
+
+    [3/4] STEP 12/13: RUN $PYCMD /output/scripts/introspect.py introspect --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
+    Creating parent directory for /tmp/src/requirements.txt
+    ---
+    python:
+    - 'netaddr  # from collection ansible.netcommon'
+    - 'netaddr>=0.10.1  # from collection ansible.utils'
+
+In the example output above, the double Python requirement for ``netaddr`` is coming from the collections
+``ansible.netcommon`` and ``ansible.utils``.
+
+The solution requires upgrading ``pip`` within the base image to a version that contains an updated dependency resolver.
+Beginning with ``pip`` version 20.3, the dependency resolver can handle duplicates requirements whose versions do not
+conflict, so that version is the minimum required. Upgrade ``pip`` in the base image from within the Execution
+Environment file by adding these lines to it:
+
+.. code-block:: yaml
+
+    additional_build_steps:
+      append_base:
+        - RUN pip3 install -U pip
+
+That will upgrade ``pip`` to the latest version within the base image. To restrict the upgrade to a specific
+version of ``pip``, alter the upgrade command to specify that version. For example:
+
+.. code-block:: yaml
+
+    additional_build_steps:
+      append_base:
+        - RUN pip3 install -U pip==20.3