Ansible request for RISC-V support

[Willsparker]

Ref: #1483 , #2121 , #2122

Using #1483 as a reference, RISC-V support needs to be added to the playbook as we have those machines. I thought it'd be best to open an issue to document the changes that need to be made from the original PR, as well as any issues that I will probably run into :-)

It may be best to look at this once #2100 has been fixed, as using mutli cores to run the playbook should significantly speed it up.

[sxa]

It may be best to look at this once #2100 has been fixed, as using mutli cores to run the playbook should significantly speed it up.

I doubt it will make a really huge % difference to be honest overall to the playbook operation since the operations are mostly sequential (some bits like compiling git/cmake/ccache if needed will see a boost), but it will make a difference to the test build of openjdk which we run afterwards.

[Willsparker]

I've started on testing the changes from the initial PR George did, however, I want to try and change the playbooks as little as possible as I do it. To this end, I've tested install gnupg2 to add the various repositories to apt, and on the RISC-V image, as long as apt-get update --fix-missing is run first, it can be installed.
Considering this is a problem specific to the QEMU VM, I may make it so that apt-get update --fix-missing is run in qemuPlaybookCheck.sh, as opposed to part of our playbook.

EDIT: An example of this can be seen in this run

[Willsparker]

The example seems encouraging! However, it appears that the repos that are added cause an issue with apt upgrade :/
I'll try just skipping them and seeing what happens..

[sxa]

Considering this is a problem specific to the QEMU VM

Which problem in particular do you think is specific to the QEMU VM?

[Willsparker]

apt installing anything results in 404 errors for the repositories - It's fixed by the apt update --fix-missing, but I assume this is specific to the QEMU image - if this were to be run on a real RISC-V machine, I'm assume they'll be more up to date than the image, and wouldn't have 404 errors. And even if they did have that error, It'd be easy to ssh and fix

[Willsparker]

New issue! The RISC-V box can't install Mercurial:

11:47:27 failed: [localhost] (item=mercurial) => {"ansible_loop_var": "item", "cache_update_time": 1619775946, "cache_updated": false, "changed": false, "item": "mercurial", "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"      install 'mercurial'' failed: E: Unable to correct problems, you have held broken packages.\n", "rc": 100, "stderr": "E: Unable to correct problems, you have held broken packages.\n" etc etc etc

From run 252.
Trying to recreate it locally now...

[Willsparker]

I was able to recreate it (by starting a fresh RISC-V QEMU VM, fixing the repos and attempting installation), weirdly, mercurial-common does exist and can be installed, but only version 5.6.1-2 , whereas we need 5.5.1-2 ...

[Willsparker]

From what I can gather, we don't actually test on RISC-V, we just build it ... If worst comes to worst, we can make RISC-V machines just skip installation of the test specific packages, but I'd rather avoid that if I can ... I think the reason this wasn't an issue when @gdams did this was because the machine he setup was Debian10, not 11 (I assume, based on the only RISC-V system we have in Jenkins)

[Willsparker]

I've removed Mercurial from the common test-tool packages, and put it in the x64 test_tool packages - When I do the proper PR, I'll make a test_tool packages task called Test_Tools_NOT_RISCV, or some such thing, as that's cleaner; but for now it will suffice for testing :-)
See QPC#257 for that test

[Willsparker]

Okay, more progress- onto failure on the docker-ce installation:

11:14:25 TASK [Docker : Install Docker for Debian <10] **********************************
11:14:35 fatal: [localhost]: FAILED! => {"changed": false, "msg": "No package matching 'docker-ce' is available"}

I'll see if there is an alternative, however, I would assume not;

EDIT: Apparently docker is there:

sudo apt-cache search docker
...
debocker - docker-powered package builder for Debian
debuerreotype - reproducible, snapshot-based Debian rootfs builder
docker-clean - simple Shell script to clean up the Docker Daemon
docker-compose - Punctual, lightweight development environments using Docker
docker-doc - Linux container runtime -- documentation
docker - transitional package
...

However, after installation, I can't run docker, I assume because docker is a transitional package in apt? Either way, I'll just skip the installation .. (See QPC#259 for the following run)

[sxa]

From what I can gather, we don't actually test on RISC-V, we just build it ...

We have test support - we just haven't fully enabled it in the pipelines yet. Mercurial isn't needed for testing AFAIK if that's the implication of the earlier comments.

Regarding docker, I'm comfortable with skipping that install on RISC-V for now.

[Willsparker]

Sorry, not sure why I didn't reply; Okay, I've done a QPC run with the main docker installation skipped: QPC#260. However, I forgot to skip the add Jenkins user to docker group tasks - which I'm not sure why they aren't tagged with jenkins - so I added those tags and rerunning QPC#263

[Willsparker]

Next issue, adoptopenjdk_install tasks. We don't produce them so:

17:11:10 TASK [adoptopenjdk_install : Install latest release if one not already installed] ***
17:11:22 FAILED - RETRYING: Install latest release if one not already installed (3 retries left).
17:11:40 FAILED - RETRYING: Install latest release if one not already installed (2 retries left).
17:11:57 FAILED - RETRYING: Install latest release if one not already installed (1 retries left).
17:12:14 fatal: [localhost]: FAILED! => {"attempts": 3, "changed": false, "msg": "Failure downloading https://api.adoptopenjdk.net/v3/binary/latest/8/ga/linux/riscv64/jdk/hotspot/normal/adoptopenjdk?project=jdk, HTTP Error 404: Not Found"}

Just need to skip the task.

EDIT:
Test here: QPC#267

[sxa]

Test here: QPC#267

IT'S GREEN!!!!!!!!!

[Willsparker]

OH MY GOD! 😂

[sxa]

I'm going to consider this closed (and push it back to May when it was completed) since we've had a clean run :-)