From abb22c01d479efc7274023282195ecf649d555fc Mon Sep 17 00:00:00 2001 From: Rogan Murley Date: Thu, 4 Aug 2022 15:45:08 +0100 Subject: [PATCH 1/3] Connection pooling recommendations --- docs/api/baseplate/clients/sqlalchemy.rst | 6 ++++++ docs/api/baseplate/observers/timeout.rst | 8 ++++++++ 2 files changed, 14 insertions(+) diff --git a/docs/api/baseplate/clients/sqlalchemy.rst b/docs/api/baseplate/clients/sqlalchemy.rst index 5fdfe38b7..b562440e4 100644 --- a/docs/api/baseplate/clients/sqlalchemy.rst +++ b/docs/api/baseplate/clients/sqlalchemy.rst @@ -52,6 +52,12 @@ request:: Configuration ------------- +.. warning:: + :py:class:`sqlalchemy.pool.QueuePool` does not handle :py:class:`~baseplate.observers.timeout.ServerTimeout` + exceptions gracefully, resulting in connection pool leaks. It is recommended + to avoid local pooling (via :py:class:`sqlalchemy.pool.NullPool`) and instead + use middleware pooling (such as `pgbouncer`) while this issue persists. + .. autoclass:: SQLAlchemySession .. autofunction:: engine_from_config diff --git a/docs/api/baseplate/observers/timeout.rst b/docs/api/baseplate/observers/timeout.rst index dafe25ea8..3df16aa68 100644 --- a/docs/api/baseplate/observers/timeout.rst +++ b/docs/api/baseplate/observers/timeout.rst @@ -14,6 +14,14 @@ upstream services are yet taken into account. taking a long time because it is doing compute-heavy actions and not yielding to the event loop it might go on longer than the allotted timeout. +.. warning:: + + The timeout mechanism may throw a :py:class:`~baseplate.observers.timeout.ServerTimeout` + exception whenever a gevent context switch occurs. If this exception is not + handled gracefully your application could get stuck in a bad state. For example, + if a `:py:class:`~baseplate.observers.timeout.ServerTimeout` exception is thrown + while releasing a connection then the connection might never be fully released. + .. versionadded:: 1.2 .. versionchanged:: 1.3.3 From 67af8773dfe1aeb31d53e8576b617283c49e999c Mon Sep 17 00:00:00 2001 From: Rogan Murley Date: Thu, 4 Aug 2022 16:54:30 +0100 Subject: [PATCH 2/3] Update docs --- docs/api/baseplate/clients/sqlalchemy.rst | 9 +++++---- docs/api/baseplate/observers/timeout.rst | 10 +++++----- 2 files changed, 10 insertions(+), 9 deletions(-) diff --git a/docs/api/baseplate/clients/sqlalchemy.rst b/docs/api/baseplate/clients/sqlalchemy.rst index b562440e4..ccdb2c828 100644 --- a/docs/api/baseplate/clients/sqlalchemy.rst +++ b/docs/api/baseplate/clients/sqlalchemy.rst @@ -53,10 +53,11 @@ Configuration ------------- .. warning:: - :py:class:`sqlalchemy.pool.QueuePool` does not handle :py:class:`~baseplate.observers.timeout.ServerTimeout` - exceptions gracefully, resulting in connection pool leaks. It is recommended - to avoid local pooling (via :py:class:`sqlalchemy.pool.NullPool`) and instead - use middleware pooling (such as `pgbouncer`) while this issue persists. + :py:class:`sqlalchemy.pool.QueuePool` does not handle `ServerTimeout` + exceptions gracefully, potentially leaking connections. It is recommended + to avoid local pooling entirely (via :py:class:`sqlalchemy.pool.NullPool`) + while this issue persists and instead rely on middleware pooling (such as + `pgbouncer`). .. autoclass:: SQLAlchemySession diff --git a/docs/api/baseplate/observers/timeout.rst b/docs/api/baseplate/observers/timeout.rst index 3df16aa68..7ca1e1081 100644 --- a/docs/api/baseplate/observers/timeout.rst +++ b/docs/api/baseplate/observers/timeout.rst @@ -16,11 +16,11 @@ upstream services are yet taken into account. .. warning:: - The timeout mechanism may throw a :py:class:`~baseplate.observers.timeout.ServerTimeout` - exception whenever a gevent context switch occurs. If this exception is not - handled gracefully your application could get stuck in a bad state. For example, - if a `:py:class:`~baseplate.observers.timeout.ServerTimeout` exception is thrown - while releasing a connection then the connection might never be fully released. + The timeout mechanism may throw a `ServerTimeout` exception whenever a gevent + context switch occurs. If this exception is not handled gracefully your + application could get stuck in a bad state. For example, if a `ServerTimeout` + exception is thrown while releasing a connection then the connection might + never be fully released. .. versionadded:: 1.2 From dd7a5478e3c5eaa7d39f70746be501e8a8b2b204 Mon Sep 17 00:00:00 2001 From: Rogan Murley Date: Thu, 4 Aug 2022 16:56:25 +0100 Subject: [PATCH 3/3] Fix spacing --- docs/api/baseplate/clients/sqlalchemy.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/api/baseplate/clients/sqlalchemy.rst b/docs/api/baseplate/clients/sqlalchemy.rst index ccdb2c828..ca025475e 100644 --- a/docs/api/baseplate/clients/sqlalchemy.rst +++ b/docs/api/baseplate/clients/sqlalchemy.rst @@ -53,6 +53,7 @@ Configuration ------------- .. warning:: + :py:class:`sqlalchemy.pool.QueuePool` does not handle `ServerTimeout` exceptions gracefully, potentially leaking connections. It is recommended to avoid local pooling entirely (via :py:class:`sqlalchemy.pool.NullPool`)