CRITICAL cursor already closed / connection already closed

[gbachRM]

Hello all! I have been receiving an error while using target-redshift where the connection is established at the start of a run, but once the data has been gathered and preparing to upload the run breaks due to the connection already being closed.

CRITICAL cursor already closed
psycopg2.OperationalError: SSL SYSCALL error: EOF detected
During handling of the above exception, another exception occurred:
psycopg2.InterfaceError: cursor already closed
During handling of the above exception, another exception occurred:
psycopg2.InterfaceError: connection already closed

If anyone could shine some light on how to resolve this issue that would be great!

[AlexanderMann]

@gbachRM this is usually some issue with SSL settings, firewalls, keep alives, and whatnot. A good way to debug this is to get a psql client setup from the same env where you're planning to run target-redshift, then debug getting the connections happy etc.

Target Redshift should expose all of the necessary configuration options to get this working. Be sure to check the older issues in here as this comes up...a lot. This might be something you need to setup a bastion for.

[connorflyn]

Hi @AlexanderMann, we are again facing the error with a loader which inherits from target-redshift.

We have tried to set the max_batch_size to as low as 50. But still face the error...

psycopg2.OperationalError: SSL connection has been closed unexpectedly 
 psycopg2.InterfaceError: cursor already closed

It would appear despite changing the max_batch_size to extremely low values, it still will not load batches of data in smaller increments!

If you could shine some light on how to resolve this issue that would be great!