"Could not Start Server" when using Podman

[andrewduss]

I've run into an issue with starting the Postgis container while using Podman on an M3 Mac, Sonoma 14.4.1. When starting any postgis image not :latest, it is unable to start:

Log snippet:

Success. You can now start the database server using:

    /usr/lib/postgresql/15/bin/pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....2024-04-05 14:47:17.891 UTC [118] LOG:  starting PostgreSQL 15.6 (Debian 15.6-1.pgdg110+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2024-04-05 14:47:17.893 UTC [118] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-04-05 14:47:17.902 UTC [126] LOG:  database system was shut down at 2024-04-05 14:47:17 UTC
. stopped waiting
pg_ctl: could not start server
Examine the log output.

When I run the exact same command using docker desktop, the image starts and runs normally. I've experienced this error with the tags 15-master, 15-3.4, 14-master, 14-3.4, and 14.3.3

My docker run command is the following

docker run \
  --name pgis \
  -e POSTGRES_PASSWORD=postgres \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_DB=postgres \
  -p 5432:5432 \
  --rm \
  postgis/postgis:15-master

and image digests

% docker image ls --digests | grep -i postgis
postgis/postgis   15-master       sha256:e7bb75ff6a637a01929fe9e871b03a14cf95e5317c13e39e58443eb0713431e6   13e5dc18959d   4 days ago     589MB
postgis/postgis   latest          sha256:21e3a909ba7a53ee35ad2b9f9ac767aa354b44cc4875895b43a7e97979fb9765   0644f71229bb   4 days ago     618MB
postgis/postgis   15-3.4          sha256:92d3db71711fc86a331218fd8f358312d1c26d638b94dce1860630a7a2f0e0eb   5da66d6b0586   4 days ago     612MB
postgis/postgis   14-master       sha256:f64aa9025759ec0a562f94fa6af57cf0aa08e85606875ca4256505a0775b37ce   46632723a746   6 months ago   569MB
postgis/postgis   14-3.3          sha256:b2ea11c9767c1b327f4c02252d9077fbbcea684d4d5b1369dc05f0eeebc82ff5   80f3732b16e2   7 months ago   595MB
postgis/postgis   <none>          sha256:2e25fd5bbbec5441d271a9608e3723e0b731bf31fa70ebbb87ccebf156672d75   44168a568a92   4 weeks ago    589MB
postgis/postgis   15-3.4-alpine   sha256:91c2f4e024b893efde3ea922189839df1a1929d43102f8073b796bb72172a0df   0f635e00750a   4 days ago     433MB

Any ideas what could be happening here?

[phillipross]

Hi @andrewduss would you be able to test with the corresponding postgres images from which the postgis images are based and see if you observe the same behavior regarding docker desktop for macos versus your podman environment? Thanks!

[ImreSamu]

This documentation at https://github.com/eriksjolund/podman-detect-option#example-3-use-detected-option-when-running-postgres-on-macos
recommends using --userns=keep-id:uid=999,gid=999 on macOS

[andrewduss]

Using the postgres:15-bullseye image and it works fine:

podman run \
  --rm \
  -it \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=password \
  -e POSTGRES_DB=mydb \
  -p 5432:5432 \
  postgres:15-bullseye

Using the PostGis image, and still not starting even after adding the --userns=keep-id:uid=999,gid=999 instructions:

podman \
  -c dflt \ *
  run \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=password \
  -e POSTGRES_DB=mydb \
  -p 5432:5432 \
  -v./data:/var/lib/postgresql/data \ *
  --userns=keep-id:uid=999,gid=999 \ *
  postgis/postgis:15-master

* = Tried with and without these options, same error as below

Log

% ./podman-gis.sh
WARNING: image platform (linux/amd64) does not match the expected platform (linux/arm64)
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    /usr/lib/postgresql/15/bin/pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....2024-04-08 17:14:46.684 UTC [86] LOG:  starting PostgreSQL 15.6 (Debian 15.6-1.pgdg110+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2024-04-08 17:14:46.687 UTC [86] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-04-08 17:14:46.699 UTC [94] LOG:  database system was shut down at 2024-04-08 17:14:46 UTC
..... stopped waiting
pg_ctl: could not start server
Examine the log output.

Podman connections:

% podman system connection list
Name        URI                                                         Identity  Default     ReadWrite
dflt        ssh://[email protected]:58822/run/user/504/podman/podman.sock  [REDACTED] true        true
dflt-root   ssh://[email protected]:58822/run/podman/podman.sock           [REDACTED] false       true

From the log output, it looks like the error is somewhere in the initialization script that PostGis uses. Is there a verbose logging option for the startup script? Curious if anyone else sees this issue using podman.

[ImreSamu]

Unfortunately, I cannot replicate the issue on my system (Ubuntu Linux + podman version 4.9.3 + X86-64 ).

Question:

• Which exact version of podman are you using? (And if it's not the latest, could you possibly test it with a newer version?)

Suggestion:

You should test whether the platform (x86 vs. amd64) might be causing the issue!
So please include the "--platform=" parameter in all tests cases!

Using the postgres:15-bullseye image and it works fine:

It's possible that in this case, you are using the default --platform=linux/arm64 image ( M3 = arm64 , apple silicon ).
Unfortunately, there isn't an arm64 version of the postgis/postgis image yet;
there is only --platform=linux/amd64, which corresponds to x86_64.

Test case suggestions (no volume, no port..):

# default postgres images / amd64(x86-64 vs. arm64(Apple silicon) 
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/amd64 docker.io/postgres:15-bullseye
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/arm64 docker.io/postgres:15-bullseye

# default postgis/postgis images  ( amd64 : debian + alpine  15-3.4 )
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/amd64 docker.io/postgis/postgis:15-3.4
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/amd64 docker.io/postgis/postgis:15-3.4-alpine

# default postgis/postgis images  ( amd64  +  latest=16-3.4 - so this should work , but please re-check!  )
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/amd64 docker.io/postgis/postgis:16-3.4
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/amd64 docker.io/postgis/postgis:16-3.4-alpine

# postgis test images linux/arm64   ( a new native arm64 images for M3 apple silicon )
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/arm64 docker.io/imresamu/postgis:15-3.4.2-bookworm
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/arm64 docker.io/imresamu/postgis:15-3.4.2-bullseye
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/arm64 docker.io/imresamu/postgis:15-3.4.2-alpine3.19
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/arm64 docker.io/imresamu/postgis:15-3.4.2-alpine3.18

# postgis test images linux/amd64  ( x86-64 )  15-3.4.2
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/amd64 docker.io/imresamu/postgis:15-3.4.2-bookworm
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/amd64 docker.io/imresamu/postgis:15-3.4.2-bullseye
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/amd64 docker.io/imresamu/postgis:15-3.4.2-alpine3.19
podman run -e POSTGRES_PASSWORD=password --pull=newer --platform=linux/amd64 docker.io/imresamu/postgis:15-3.4.2-alpine3.18

Other test ideas:

You might also want to test using the --privileged flag to rule out any permission issues.

• https://docs.podman.io/en/latest/markdown/podman-run.1.html#privileged

And other test ideas (from the PostGIS Matrix chat):

• Can you replicate the problem on an AWS Apple Silicon instance?