Migrate script broken on php8.2 and outdated cURL cacert.pem

[czephyr]

Hi,

while building the latest image tagged 2.6.0-php8.2 we encountered two issues.
First one was with the migrate script which bring ups the error:

WARNING! You are about to execute a database migration that could result in schema changes and data lost. Are you sure you wish to continue? (y/n)y
Migrating up to 20160407194955 from 20160407193445
  ++ migrating 20160407194955
Creating new sessions table
     -> SELECT 1
  ++ migrated (0.14s)
Migration 20160407194955 failed during Post-Checks. Error There is no active transaction
  [PDOException]
  There is no active transaction

I'm not keen on PHP but a quick search seems to bring up some info:
https://stackoverflow.com/questions/67533806/there-is-no-active-transaction-in-doctrine-migrations

We circumvented the error by importing the database dump from a previous CalDAV instance we had already running.

---

Second problem is the outdated cacert.pem in the image.
The server answers with an error just after the login, this is in the logs:

[2023-03-16 15:11:10] agendav.CRITICAL: GuzzleHttp\Exception\RequestException: cURL error 77: error setting certificate verify locations:  CAfile: /etc/ssl/certs/cacert.pem CApath: /etc/ssl/certs (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) (uncaught exception) at /var/www/agendav/web/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php 
line 201 {"exception":"[object] (GuzzleHttp\\Exception\\RequestException(code: 0): cURL error 77: error setting certificate verify locations:  CAfile: /etc/ssl/certs/cacert.pem CApath: /etc/ssl/certs (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) at /var/www/agendav/web/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php:201)"} []

We circumvented this error by repacking the docker image in our dockerfile adding:

COPY files/cacert.pem /etc/ssl/certs/cacert.pem
RUN chmod 777 /etc/ssl/certs/cacert.pem

The cacert.pem file was obtained as mentioned here:
https://stackoverflow.com/questions/55204210/curl-error-77-error-setting-certificate-verify-locations-cafile

[nagimov]

@eugene-davis could you troubleshoot the issue with the db migration? I'm guessing this mysql -> sqlite change bccf36b

[eugene-davis]

@czephyr Can you give me some details on how you are running the build?

Especially:

• What platform?
• What builder (e.g. Docker, Podman, Kaniko...)?
• Did you just run the build command in the repository after checking out 2.6.0-php8.2?
• Are you mounting in anything during the build or using any special flags?

I just ran the build on amd64 with Podman and it did not hit the migration issue - however I think I saw that error when I tried to transition to a newer version of PHP before switching to sqlite.

[czephyr]

Ah, sorry for leaving those out.
We're building on Kubernetes with Docker and trying to pair with a mysql instance.
We also installed the pdo_mysql adding it to the custom Dockerfile.

We're pulling with:
FROM ghcr.io/nagimov/agendav-docker:2.6.0-php8.2

No special flags, just pulling the image from Kubernetes.

[eugene-davis]

So if I understand correctly, you are building a custom image using ghcr.io/nagimov/agendav-docker:2.6.0-php8.2 as a base, connecting it to an external MySQL database and then running migrations on that?

This image was built to be stateless, bypassing the need entirely for an external database.

I think cdb7a53 was still working with MySQL, but although that runs the latest AgenDAV, it is running on the PHP 7 line - which is already EOL.
Even if it does work during the build of this image, I can't promise that it will work successfully with your existing database.

Realistically, I think this is outside of the scope of what can be supported here (@nagimov you are of course free to tell me I'm wrong).

I can make a couple of suggestions though:

1. See if you can do without the external database
2. Check your mysql configuration against the doctrine docs, maybe see if you can use a different mysql driver
3. Open a ticket with https://github.com/agendav/agendav

[czephyr]

Yes that's exactly what we're doing.

I understand the principle of statelessness but would you mind expanding on the reason this image would not support an external database?

We decided to support it with an external database since we're running this service in internal production for our own company and we decided against using sqlite. What's the point of having this image if it can't be run with a production ready database system?

[czephyr]

bump @nagimov; any info about the curl cacert?

[nagimov]

I understand the principle of statelessness but would you mind expanding on the reason this image would not support an external database?

@czephyr I agree with @eugene-davis, this is outside of our scope. Agendav is just a wrapper around existing caldav deployment. Persistency should be taken cared of by the caldav backend. There's simply no reason to preserve anything within agendav instances. As such, "non-production-readiness" of sqlite is not a real problem: you could always re-create the instance without any loss of data. I would encourage you to get rid of the unnecessary mysql dependence (which, btw, is an additional point of failure) and simplify your k8s deployment. IMO, running an EOL version of php in production is worse than backing a non-persistent state with sqlite.

Regarding the cert issue, I'm not sure what causes this. The image pulls the https://curl.se/ca/cacert.pem file and places it to /etc/ssl/certs, so pretty much the same as your fix

COPY files/cacert.pem /etc/ssl/certs/cacert.pem
RUN chmod 777 /etc/ssl/certs/cacert.pem

Any chance that your cert file was preserved from a previous instance?

[czephyr]

Regarding the cert issue, I'm not sure what causes this. The image pulls the https://curl.se/ca/cacert.pem file and places it to /etc/ssl/certs, so pretty much the same as your fix

COPY files/cacert.pem /etc/ssl/certs/cacert.pem
RUN chmod 777 /etc/ssl/certs/cacert.pem

@nagimov The fix pulls from http://curl.haxx.se/ca/cacert.pem, not exactly the same, but not sure what's different. We re-tried the plain image just now, without the additional cacert.pem it errors out.

Agendav is just a wrapper around existing caldav deployment. Persistency should be taken cared of by the caldav backend. There's simply no reason to preserve anything within agendav instances. As such, "non-production-readiness" of sqlite is not a real problem: you could always re-create the instance without any loss of data.

I see, thanks for the explanation. So everything we see in the agendavdb is just pulled from the caldav backend? The sqlite is only used as a cache?

[nagimov]

@nagimov The fix pulls from http://curl.haxx.se/ca/cacert.pem, not exactly the same, but not sure what's different. We re-tried the plain image just now, without the additional cacert.pem it errors out.

These are identical files:

$ wget -q https://curl.se/ca/cacert.pem -O cert1
$ wget -q https://curl.haxx.se/ca/cacert.pem -O cert2
$ md5sum cert1 cert2
e7cf471ba7c88f4e313f492a76e624b3  cert1
e7cf471ba7c88f4e313f492a76e624b3  cert2

Could you try adding chmod into your Dockerfile?

I see, thanks for the explanation. So everything we see in the agendavdb is just pulled from the caldav backend? The sqlite is only used as a cache?

You could view it functionally as a cache, yes (although agendav devs might disagree). The point is that the sqlite db could be wiped without any loss of data, so it doesn't have to be persistent.

[czephyr]

@nagimov

Could you try adding chmod into your Dockerfile?

That works, the image native cacert.pem can't be accessed from the user www-data which is completely breaking for the image, what is the best way to approach this fix?

Also btw, just for documenting, we considered using sqlite but it currently runs into this other breaking error (the service goes unavailable for a minute every time an user tries to save) agendav/agendav#306

Using mysql works, if only for agendav/agendav#303, which seems to get fixed by the suggested fix in the issue.

[nagimov]

@czephyr please test 80b526b for certificate issues

[czephyr]

@nagimov The cacert issue is solved.