HIVE-28597: Upgrade to 8.4.3 LTS version of MySQL

[KiranVelumuri]

What changes were proposed in this pull request?

Upgrade MySQL version

Why are the changes needed?

HIVE-28597

Does this PR introduce any user-facing change?

No

Is the change a dependency upgrade?

No

How was this patch tested?

mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=qt_database_mysql.q -pl itests/qtest -Pitests

[deniskuzZ]

LGTM, pending green build

[KiranVelumuri]

In Jenkins, it is failing to download the newer image due to missing signature key. This seems like a client side issue due to some restriction on downloading Docker image. I checked that the earlier changes to mysql version only updated the version, and nothing related to docker. Could you please help with this environmental issue? Thanks.

2024-10-30T04:26:21,536  INFO [main] externalDB.AbstractExternalDB: Going to run: docker run --rm --name qtestExternalDB-MySQLExternalDB -p 3306:3306 -e MYSQL_ROOT_PASSWORD=qtestpassword -e MYSQL_DATABASE=qtestDB -d mysql:8.4.3
2024-10-30T04:26:22,298  INFO [main] externalDB.AbstractExternalDB: Result lines#: 0(stdout);127(stderr)
2024-10-30T04:26:22,298  INFO [main] externalDB.AbstractExternalDB: Stdout from proc: 
2024-10-30T04:26:22,298  INFO [main] externalDB.AbstractExternalDB: Stderr from proc: Unable to find image 'mysql:8.4.3' locally
docker: Error response from daemon: missing signature key.
See 'docker run --help'.

Update 1: checking if docker client version is the issue
You can not pull images with a docker version older than 18.06 since the registry started to use the v2 schema.
Output: Docker version 20.10.5+dfsg1, build 55c4c88

Update 2: checking if docker server is at older version and client is at newer version

2024-11-04T03:25:21,134  INFO [main] externalDB.AbstractExternalDB: Going to run: docker version
2024-11-04T03:25:21,348  INFO [main] externalDB.AbstractExternalDB: Result lines#: 512(stdout);0(stderr)
2024-11-04T03:25:21,349  INFO [main] externalDB.AbstractExternalDB: Stdout from proc: Client:
 Version:           20.10.5+dfsg1
 API version:       1.37
 Go version:        go1.15.15
 Git commit:        55c4c88
 Built:             Mon May 30 18:34:49 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          18.05.0-ce
  API version:      1.37 (minimum version 1.12)
  Go version:       go1.10.1
  Git commit:       f150324
  Built:            Wed May  9 22:20:42 2018
  OS/Arch:          linux/amd64
  Experimental:     false

From this, the older server might be causing this issue. @deniskuzZ Could you please help on how to upgrade the docker server version on these VMs?

[deniskuzZ]

In Jenkins, it is failing to download the newer image due to missing signature key. This seems like a client side issue due to some restriction on downloading Docker image. I checked that the earlier changes to mysql version only updated the version, and nothing related to docker. Could you please help with this environmental issue? Thanks.

2024-10-30T04:26:21,536  INFO [main] externalDB.AbstractExternalDB: Going to run: docker run --rm --name qtestExternalDB-MySQLExternalDB -p 3306:3306 -e MYSQL_ROOT_PASSWORD=qtestpassword -e MYSQL_DATABASE=qtestDB -d mysql:8.4.3
2024-10-30T04:26:22,298  INFO [main] externalDB.AbstractExternalDB: Result lines#: 0(stdout);127(stderr)
2024-10-30T04:26:22,298  INFO [main] externalDB.AbstractExternalDB: Stdout from proc: 
2024-10-30T04:26:22,298  INFO [main] externalDB.AbstractExternalDB: Stderr from proc: Unable to find image 'mysql:8.4.3' locally
docker: Error response from daemon: missing signature key.
See 'docker run --help'.

Update 1: checking if docker client version is the issue You can not pull images with a docker version older than 18.06 since the registry started to use the v2 schema. Output: Docker version 20.10.5+dfsg1, build 55c4c88

Update 2: checking if docker server is at older version and client is at newer version

2024-11-04T03:25:21,134  INFO [main] externalDB.AbstractExternalDB: Going to run: docker version
2024-11-04T03:25:21,348  INFO [main] externalDB.AbstractExternalDB: Result lines#: 512(stdout);0(stderr)
2024-11-04T03:25:21,349  INFO [main] externalDB.AbstractExternalDB: Stdout from proc: Client:
 Version:           20.10.5+dfsg1
 API version:       1.37
 Go version:        go1.15.15
 Git commit:        55c4c88
 Built:             Mon May 30 18:34:49 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          18.05.0-ce
  API version:      1.37 (minimum version 1.12)
  Go version:       go1.10.1
  Git commit:       f150324
  Built:            Wed May  9 22:20:42 2018
  OS/Arch:          linux/amd64
  Experimental:     false

From this, the older server might be causing this issue. @deniskuzZ Could you please help on how to upgrade the docker server version on these VMs?

i don't have access to the VMs, but I think @abstractdog can help

[abstractdog]

In Jenkins, it is failing to download the newer image due to missing signature key. This seems like a client side issue due to some restriction on downloading Docker image. I checked that the earlier changes to mysql version only updated the version, and nothing related to docker. Could you please help with this environmental issue? Thanks.

2024-10-30T04:26:21,536  INFO [main] externalDB.AbstractExternalDB: Going to run: docker run --rm --name qtestExternalDB-MySQLExternalDB -p 3306:3306 -e MYSQL_ROOT_PASSWORD=qtestpassword -e MYSQL_DATABASE=qtestDB -d mysql:8.4.3
2024-10-30T04:26:22,298  INFO [main] externalDB.AbstractExternalDB: Result lines#: 0(stdout);127(stderr)
2024-10-30T04:26:22,298  INFO [main] externalDB.AbstractExternalDB: Stdout from proc: 
2024-10-30T04:26:22,298  INFO [main] externalDB.AbstractExternalDB: Stderr from proc: Unable to find image 'mysql:8.4.3' locally
docker: Error response from daemon: missing signature key.
See 'docker run --help'.

Update 1: checking if docker client version is the issue You can not pull images with a docker version older than 18.06 since the registry started to use the v2 schema. Output: Docker version 20.10.5+dfsg1, build 55c4c88
Update 2: checking if docker server is at older version and client is at newer version

2024-11-04T03:25:21,134  INFO [main] externalDB.AbstractExternalDB: Going to run: docker version
2024-11-04T03:25:21,348  INFO [main] externalDB.AbstractExternalDB: Result lines#: 512(stdout);0(stderr)
2024-11-04T03:25:21,349  INFO [main] externalDB.AbstractExternalDB: Stdout from proc: Client:
 Version:           20.10.5+dfsg1
 API version:       1.37
 Go version:        go1.15.15
 Git commit:        55c4c88
 Built:             Mon May 30 18:34:49 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          18.05.0-ce
  API version:      1.37 (minimum version 1.12)
  Go version:       go1.10.1
  Git commit:       f150324
  Built:            Wed May  9 22:20:42 2018
  OS/Arch:          linux/amd64
  Experimental:     false

From this, the older server might be causing this issue. @deniskuzZ Could you please help on how to upgrade the docker server version on these VMs?

i don't have access to the VMs, but I think @abstractdog can help

it should work I believe...anyway, I can take a look this week
this needs a running PR when I can enter into the container, I'll try
worst case is to rebuild the precommit images by contributing to another repo which holds the Dockerfiles

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
1 Security Hotspot
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[abstractdog]

guys, I've set this blocked on HIVE-28629, which is about to bump the dind image version

[KiranVelumuri]

@abstractdog Could you please prioritize HIVE-28629 since this is blocked on it? Thanks.