Not honoring hugepages setting during initdb causes DB crash

[howels]

Overview

On an OS where hugepages are enabled, if no hugepages resources are assigned in Kubernetes and the postgres instance is set to hugepages = off in the config then one would assume that the DB would not use hugepages.
However, because the initdb process uses postgresql.conf.sample or postgresql.conf.template instead of the actual specified configuration the applied setting is actually hugepages = try during initdb.
In these case the initdb phase will attempt to allocate hugepages which are available in the OS, but it will be denied access by Kubernetes and fail.

Environment

• Environment: Rancher RKE2
• PGO: 5.2
• RKE2: 1.24/1.25

Steps to Reproduce

Run a DB with no hugepages resources assigned on a OS with hugepages and with hugepages = off set in postgres conf.

Expected

When hugepages are disabled they should not be used, during initdb or otherwise. The inability to specify hugepages forces all DBs to use hugepages on any system where they are enabled, even if the hugepages are intended for other applications and not available to the DB Pods.

Actual

Postgres crashes during initdb after speculatively attempting to access hugepages and being denied by Kubernetes policies.

Additional Information
Issue has persisted for some time and the implementation goes against the spirit of the configuration options provided. Exceedingly awkward workaround such as attempting to mount configmaps over postgresql.conf.sample are not suitable for ongoing long-term usage.

[howels]

Note that we are aware of #3039 but this is a workaround not a solution. HugePages should only be used by apps which explicity require hugepages.

[howels]

Bug also mentioned in #3421

[David-Angel]

The workaround to enable hugepages isn't going to work when you are required to disable hugepages.
We have a requirement not to use hugepages which can't be disabled without altering crunchydata code.

This file needs to change to turn it off.
./usr/pgsql-14/share/postgresql.conf.sample

initdb uses that file instead of the standard config file.
It would be very nice if crunchydata exposed a way of doing this through the "postgrescluster".

[howels]

Can we please get a response on this?

[David-Angel]

Here is a possible fix:
CrunchyData/crunchy-containers@master...David-Angel:crunchy-containers:patch-1

This has become a critical issue for us as we are unable to build the crunchy containers to make the customization.

That fix might be valid in every case.
initdb uses "postgresql.conf.sample" and shuts down before postgres is started.
postgres may be able to honor the standard configuration when it is started.

[David-Angel]

Others with the same issue.
#3039
#2258
#3126
#3421

[David-Angel]

I've verified with PostgreSQL support.
To disable huge_pages crunchydata needs to update the sample config before calling initdb.

Conversation with PostgreSQL
https://www.postgresql.org/message-id/flat/17757-dbdfc1f1c954a6db%40postgresql.org

[David-Angel]

Here is another solution:
#3555

[David-Angel]

This may be resolved in the latest build by Crunchydata as they updated their documentation to say it is now disabled by default.
https://access.crunchydata.com/documentation/postgres-operator/latest/guides/huge-pages/

[dsessler7]

As @David-Angel has indicated, this issue has been resolved in our latest releases: 5.4.0, 5.3.2, 5.2.3, and 5.1.6.