aixPb: Add steps to install dnf and deprecate yum for AIX 7.2 and higher #3271

Haroon-Khel · 2023-11-29T11:53:17Z

commit message has one of the standard prefixes
faq.md updated if appropriate
other documentation is changed or added (if applicable)
playbook changes run through VPC or QPC (if you have access)
VPC/QPC not applicable for this PR
for inventory.yml changes, bastillion/nagios/jenkins updated accordingly

ref #3142 (comment)

Haroon-Khel · 2023-11-29T14:20:21Z

As per @aixtools's comments in #3149 (comment) I am also in favour of using dnf only and deprecating yum

@sxa What do you think?

Haroon-Khel · 2023-11-29T14:24:25Z

@aixtools I am not too familiar with

infrastructure/ansible/playbooks/AdoptOpenJDK_AIX_Playbook/roles/yum/tasks/main.yml

Line 15 in 1e3dfa0

- name: Remove conflicting packages before install yum

Do you think it needs to be included in the dnf install?

sxa · 2023-11-29T15:42:07Z

As per @aixtools's comments in #3149 (comment) I am also in favour of using dnf only and deprecating yum

@sxa What do you think?

Given that we no longer build on AIX 71 and therefore cannot run on it, that sounds reasonable to me.

sxa

These removals make me shudder a little, but if we can add a checksum into the toolbox download that would be good t avoid potential tampering ... Not sure how often that script gets updated though, so we may have to keep updating it.

ansible/playbooks/AdoptOpenJDK_AIX_Playbook/main.yml

ansible/playbooks/AdoptOpenJDK_AIX_Playbook/roles/dnf/tasks/main.yml

sxa

LGTM

Haroon-Khel · 2023-11-30T12:16:45Z

Going to test this on https://ci.adoptium.net/computer/test-osuosl-aix72-ppc64-5/, its yum will be upgraded to dnf

Haroon-Khel · 2023-11-30T12:43:58Z

Despite the docs saying it needs openssl 1.0.2.2001 and higher (which test-osuosl-aix72-ppc64-5 has)

https://community.ibm.com/community/user/power/blogs/sangamesh-mallayya1/2021/05/28/dnf-is-now-available-on-aix-toolbox?CommunityKey=10c1d831-47ee-4d92-a138-b03f7896f7c9&tab=recentcommunityblogsdashboard

dnf works on AIX 7.1 TL3 and higher versions.
Openssl 1.0.2.2001 and higher version needs to be installed.

the script is demanding openssl 1.1.x or higher

TASK [dnf : Install dnf] *******************************************************
fatal: [test-osuosl-aix72-ppc64-5]: FAILED! => {"changed": true, "cmd": ["/tmp/dnf_aixtoolbox.sh", "-y"], "delta": "0:00:00.733893", "end": "2023-11-30 12:34:36.752712", "msg": "non-zero return code", "rc": 1, "start": "2023-11-30 12:34:36.018819", "stderr": "", "stderr_lines": [], "stdout": "Please install openssl 1.1.x and higher version.\nYou can download and install latest openssl from AIX web download site\nhttps://www-01.ibm.com/marketing/iwm/platform/mrs/assets?source=aixbp", "stdout_lines": ["Please install openssl 1.1.x and higher version.", "You can download and install latest openssl from AIX web download site", "https://www-01.ibm.com/marketing/iwm/platform/mrs/assets?source=aixbp"]}

The openssl role ensures 1.0.2 is on the machine, do we have anything that depends on this version or could this be bumped to 1.1.x? @sxa @aixtools

On our only 7.3 machine, on which dnf is installed, its openssl version is 3.0.7

root@adopt01:[/root]openssl version
OpenSSL 3.0.7 1 Nov 2022 (Library: OpenSSL 3.0.7 1 Nov 2022)

sxa · 2023-11-30T15:07:34Z

Please install openssl 1.1.x and higher version.\nYou can download and install latest openssl from AIX web download site\nhttps://www-01.ibm.com/marketing/iwm/platform/mrs/assets?source=aixbp

OK so the important part here is thius message:
Please install openssl 1.1.x and higher version. You can download and install latest openssl from AIX web download site https://www-01.ibm.com/marketing/iwm/platform/mrs/assets?source=aixbp`

Since this is an official IBM AIX download I would expect that it should be safe to install this, so I would suggest trying it on one of the AIX 7.2 machines if we haven't done so on any of the machines already.

But good to know it shouldn't be a problem on the new AIX 7.3 machine.

Haroon-Khel · 2023-12-01T13:24:59Z

Installed openssl3 on test-osuosl-aix72-ppc64-5. Doesnt look like there was a problem

root@adopt05:[/root/openssl3/openssl-3.0.10.1000]openssl version
OpenSSL 3.0.10 1 Aug 2023 (Library: OpenSSL 3.0.10 1 Aug 2023)

Haroon-Khel · 2023-12-01T13:45:07Z

Its having trouble installing ca-certificates-2023.2.60-0.aix7.1.ppc.rpm which is included in the dnf bundle. A lot of conflicting files between ca-certificates-2023.2.60-0.aix7.1.ppc.rpm and the ca certs thats already installed, ca-certificates-2021.2.52-1.ppc. For example

file /opt/freeware/bin/update-ca-bundles from install of ca-certificates-2023.2.60-0.ppc conflicts with file from package ca-certificates-2021.2.52-1.ppc
        file /opt/freeware/etc/ssl/certs/ACCVRAIZ1.crt from install of ca-certificates-2023.2.60-0.ppc conflicts with file from package ca-certificates-2021.2.52-1.ppc
        file /var/ssl/certs/ACCVRAIZ1.crt from install of ca-certificates-2023.2.60-0.ppc conflicts with file from package ca-certificates-2021.2.52-1.ppc
        file /opt/freeware/etc/ssl/certs/AC_RAIZ_FNMT-RCM.crt from install of ca-certificates-2023.2.60-0.ppc conflicts with file from package ca-certificates-2021.2.52-1.ppc
        file /var/ssl/certs/AC_RAIZ_FNMT-RCM.crt from install of ca-certificates-2023.2.60-0.ppc conflicts with file from package ca-certificates-2021.2.52-1.ppc
        file /opt/freeware/etc/ssl/certs/AC_RAIZ_FNMT-RCM_SERVIDORES_SEGUROS.crt from install of ca-certificates-2023.2.60-0.ppc conflicts with file from package ca-certificates-2021.2.52-1.ppc
        file /var/ssl/certs/AC_RAIZ_FNMT-RCM_SERVIDORES_SEGUROS.crt from install of ca-certificates-2023.2.60-0.ppc conflicts with file from package ca-certificates-2021.2.52-1.ppc
        file /opt/freeware/etc/ssl/certs/ANF_Secure_Server_Root_CA.crt from install of ca-certificates-2023.2.60-0.ppc conflicts with file from package ca-certificates-2021.2.52-1.ppc
        file /var/ssl/certs/ANF_Secure_Server_Root_CA.crt from install of ca-certificates-2023.2.60-0.ppc conflicts with file from package ca-certificates-2021.2.52-1.ppc

(without posting the whole log)

Haroon-Khel · 2023-12-01T14:34:40Z

Updating ca-certificates to ca-certificates-2021.2.52-3 I was able to fix the above error. Dnf successfully installed

Haroon-Khel · 2023-12-01T14:35:54Z

Need to specify a python 3 interpreter

failed: [test-osuosl-aix72-ppc64-5] (item=bash) => {"ansible_loop_var": "item", "changed": false, "item": "bash", "msg": 
"Could not import the dnf python module using /usr/bin/python (2.7.18 (default, Dec 15 2022, 05:12:43) [GCC 8.3.0]). Please
 install `python3-dnf` or `python2-dnf` package or ensure you have specified the correct ansible_python_interpreter. 
(attempted ['/usr/libexec/platform-python', '/usr/bin/python3', '/usr/bin/python2', '/usr/bin/python'])", "results": []}

Haroon-Khel · 2023-12-01T14:40:45Z

The package yum-utils needs to be removed before running dnf update to resolve this error
/opt/freeware/bin/yum conflicts between attempted installs of yum-3.4.3-8.noarch and dnf-4.2.17-32_51.ppc

… test

Haroon-Khel · 2023-12-01T15:51:50Z

Changing the python interpreter to ansible_python_interpreter: /opt/freeware/libexec/python3 for whenever the dnf module is used works

more info here https://community.ibm.com/community/user/power/discussion/dnf-works-but-unable-to-import-dnf-python-module

Haroon-Khel · 2023-12-01T16:01:22Z

Its having trouble installing some packages

failed: [test-osuosl-aix72-ppc64-5] (item=freetype2-devel-2.8-1) => {"ansible_loop_var": "item", "changed": false, "failures": ["freetype2-devel-2.8-1 All matches were filtered out by exclude filtering for argument: freetype2-devel-2.8-1"], "item": "freetype2-devel-2.8-1", "msg": "Failed to install some of the specified packages", "rc": 1, "results": []}

failed: [test-osuosl-aix72-ppc64-5] (item=autoconf-2.69-1) => {"ansible_loop_var": "item", "changed": false, "failures": ["autoconf-2.69-1 All matches were filtered out by exclude filtering for argument: autoconf-2.69-1"], "item": "autoconf-2.69-1", "msg": "Failed to install some of the specified packages", "rc": 1, "results": []}

Both of these packages are on the "exclude" list, that is they are not to be updated because we want to maintain a particular version. But I dont know why this would cause an error for ansible

failed: [test-osuosl-aix72-ppc64-5] (item=gnupg2-2.0.30) => {"ansible_loop_var": "item", "changed": false, "failures": [], "item": "gnupg2-2.0.30", "msg": "Depsolve Error occured: \n Problem: problem with installed package gpgme-1.13.1-101.ppc\n  - package gpgme-1.13.1-101.ppc requires gnupg2 >= 2.2.23, but none of the providers can be installed\n  - package gpgme-1.13.1-100.ppc requires gnupg2 >= 2.2.23, but none of the providers can be installed\n  - cannot install both gnupg2-2.0.30-2.ppc and gnupg2-2.2.35-1.ppc\n  - cannot install both gnupg2-2.2.23-1.ppc and gnupg2-2.0.30-2.ppc\n  - cannot install both gnupg2-2.2.35-1.ppc and gnupg2-2.0.30-2.ppc\n  - cannot install the best candidate for the job", "rc": 1, "results": []}

Haroon-Khel · 2023-12-01T19:21:44Z

Lets see what adding a disable_excludes: all does. It should still install the excluded packages if theyre not present, and do nothing if theyre present since in the install list the autoconf and freetype2-devel include their version numbers

Haroon-Khel · 2023-12-01T19:23:54Z

TASK [dnf : Install packages] **************************************************
changed: [test-osuosl-aix72-ppc64-5] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: fontconfig-devel-2.11.95-4.ppc", "Installed: expat-devel-2.5.0-1.ppc", "Installed: fontconfig-2.11.95-4.ppc"]}

Looks good, seems to have only updated some dependencies. Plus installing the packages as a list instead of using with_items makes the install ALOT faster

Haroon-Khel · 2023-12-01T19:24:35Z

Same problem with the cmake install

TASK [dnf : Install cmake 3.14.3 (See https://github.com/AdoptOpenJDK/openjdk-build/issues/2492)] ***
fatal: [test-osuosl-aix72-ppc64-5]: FAILED! => {"changed": false, "failures": ["cmake-3.14.3 All matches were filtered out by exclude filtering for argument: cmake-3.14.3"], "msg": "Failed to install some of the specified packages", "rc": 1, "results": []}

Should be made more idempotent

Haroon-Khel · 2023-12-01T19:29:46Z

Lets see what adding a disable_excludes: all does. It should still install the excluded packages if theyre not present, and do nothing if theyre present since in the install list the autoconf and freetype2-devel include their version numbers

Added this to the cmake install. Should be fine if theres no cmake-3.14.3 present. Should do nothing if it is present

Haroon-Khel · 2023-12-01T19:34:40Z

https://awx2.adoptopenjdk.net/#/jobs/playbook/2256?job_search=page_size:20;order_by:-finished;not__launch_type:sync

Looks good

Haroon-Khel · 2023-12-01T22:42:55Z

Id say this is good to merge

add dnf role, deprecate yum role

b97fd84

github-actions bot added ansible os:aix labels Nov 29, 2023

Haroon-Khel added 3 commits November 29, 2023 14:09

Add check for python3, comments and other steps

b501c32

ad dnf tags to role

40610ff

linter

ffd6be0

Haroon-Khel marked this pull request as ready for review November 29, 2023 14:13

Haroon-Khel requested review from aixtools and removed request for aixtools November 29, 2023 14:19

Haroon-Khel requested review from aixtools and sxa November 29, 2023 14:20

include code about removing unnecessary packages

0ba33ea

sxa requested changes Nov 29, 2023

View reviewed changes

ansible/playbooks/AdoptOpenJDK_AIX_Playbook/main.yml Show resolved Hide resolved

ansible/playbooks/AdoptOpenJDK_AIX_Playbook/roles/dnf/tasks/main.yml Show resolved Hide resolved

add checksum to dnf install script

95b7152

sxa approved these changes Nov 30, 2023

View reviewed changes

Haroon-Khel added 2 commits November 30, 2023 12:24

debug

2ab2084

change permissions of dnf script to execute

982d1be

debug

6ba97fa

remove debug, remove yum-utils after dnf install and before dnf update

9e5d631

Haroon-Khel added 3 commits December 1, 2023 15:30

modify ansible_python_interpreter as a test

c4009db

remove ansible_python_interpreter from main and have as var for test

26644cf

change ansible_python_interpreter to /opt/freeware/libexec/python3 to…

4b8b951

… test

move dnf module tasks into own ansible_interpreter block

ec47488

Haroon-Khel added 3 commits December 1, 2023 19:05

ignore gnupg2-2.0.30 for now

124d5d2

move list of packages outside of with_items, as list

3f035c1

add disable_excludes

a1e9147

add disable_excludes to cmake install

df9538e

reduce time in remove packages task

52fff30

Haroon-Khel mentioned this pull request Dec 1, 2023

Add steps to install OpenSSL 3 on AIX 7.2 machines #3274

Closed

karianna approved these changes Dec 2, 2023

View reviewed changes

karianna merged commit 649c7a7 into adoptium:master Dec 2, 2023
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

aixPb: Add steps to install dnf and deprecate yum for AIX 7.2 and higher #3271

aixPb: Add steps to install dnf and deprecate yum for AIX 7.2 and higher #3271

Haroon-Khel commented Nov 29, 2023 •

edited

Loading

Haroon-Khel commented Nov 29, 2023 •

edited

Loading

Haroon-Khel commented Nov 29, 2023

sxa commented Nov 29, 2023

sxa left a comment

sxa left a comment

Haroon-Khel commented Nov 30, 2023 •

edited

Loading

Haroon-Khel commented Nov 30, 2023 •

edited

Loading

sxa commented Nov 30, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023 •

edited

Loading

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

aixPb: Add steps to install dnf and deprecate yum for AIX 7.2 and higher #3271

aixPb: Add steps to install dnf and deprecate yum for AIX 7.2 and higher #3271

Conversation

Haroon-Khel commented Nov 29, 2023 • edited Loading

Haroon-Khel commented Nov 29, 2023 • edited Loading

Haroon-Khel commented Nov 29, 2023

sxa commented Nov 29, 2023

sxa left a comment

Choose a reason for hiding this comment

sxa left a comment

Choose a reason for hiding this comment

Haroon-Khel commented Nov 30, 2023 • edited Loading

Haroon-Khel commented Nov 30, 2023 • edited Loading

sxa commented Nov 30, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023 • edited Loading

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Dec 1, 2023

Haroon-Khel commented Nov 29, 2023 •

edited

Loading

Haroon-Khel commented Nov 29, 2023 •

edited

Loading

Haroon-Khel commented Nov 30, 2023 •

edited

Loading

Haroon-Khel commented Nov 30, 2023 •

edited

Loading

Haroon-Khel commented Dec 1, 2023 •

edited

Loading