test_pgdata_import_smoke timeouts

[jcsp]

https://neon-github-public-dev.s3.amazonaws.com/reports/main/12253951655/index.html#testresult/3359205196681dba/retries

test_runner/regress/test_import_pgdata.py:305: in test_pgdata_import_smoke
    validate_vanilla_equivalence(br_initdb_endpoint)
test_runner/regress/test_import_pgdata.py:114: in validate_vanilla_equivalence
    assert ep.safe_psql("select count(*), sum(data::bigint)::bigint from t") == [
test_runner/fixtures/neon_fixtures.py:283: in safe_psql
    return self.safe_psql_many([query], **kwargs)[0]
test_runner/fixtures/neon_fixtures.py:298: in safe_psql_many
    cur.execute(query)
/home/nonroot/.pyenv/versions/3.11.10/lib/python3.11/encodings/utf_8.py:15: in decode
    def decode(input, errors='strict'):
E   Failed: Timeout >300.0s

[jcsp]

Typical runtimes are ~210s

[jcsp]

Timeout is always during the last validate_vanilla_equivalent call onto the br-initdb-ro endpoint.

We give it a basebackup successfully:

2024-12-10T10:46:37.657219Z  INFO page_service_conn_main{peer_addr=127.0.0.1:40632}:process_query{tenant_id=42884b4d19e567b90afaf354195a9e50 timeline_id=b094f8a43aa6d5e6a4d7f947a7dbcb0b}:handle_basebackup_request{lsn=None prev_lsn=None full_backup=false shard_id=0000}: basebackup complete lsn_await_millis=0 basebackup_millis=27

Endpoint logs don't say much:

PG:2024-12-10 10:46:37.880 GMT [142957] LOG:  [NEON_SMGR] [shard 0] libpagestore: connected to 'postgresql://no_user@localhost:15060' with protocol version 2
2024-12-10T10:46:38.000833Z  INFO [NEON_EXT_STAT] {"extensions":[{"extname":"plpgsql","versions":["1.0"],"n_databases":2}]}

... time passes...

PG:2024-12-10 10:47:37.812 GMT [154386] LOG:  [NEON_SMGR] [shard 0] libpagestore: connected to 'postgresql://no_user@localhost:15060' with protocol version 2
PG:2024-12-10 10:47:50.205 GMT [142934] LOG:  received fast shutdown request

[jcsp]

The test's runtime is dominated by the simple postgres query that sums a bunch of integers. The test intentionally bloats out the data to take a lot of pages (~300k), and the reads are not batched because tests run with effective_io_concurrency=1, so we end up CPU bottlenecked on the pageserver, making this test rather sensitive to background CPU load.