tests: WIP: MITM proxy between pageserver and compute for fault testing

[hlinnaka]

This doesn't do much yet, but provides the basic man-in-the-middle proxy that can be used to inject faults in the compute <-> compute communication. I will use this to write test various scenarios, to stress the error handling and retry logic in the compute.

[github-actions]

6410 tests run: 6130 passed, 6 failed, 274 skipped (full report)

---

Failures on Postgres 17

• test_compute_pageserver_connection_stress2: release-x86-64, debug-x86-64, release-arm64, release-arm64

Failures on Postgres 16

• test_compute_pageserver_connection_stress2: release-arm64

Failures on Postgres 14

• test_compute_pageserver_connection_stress2: release-arm64

# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_compute_pageserver_connection_stress2[release-pg14] or test_compute_pageserver_connection_stress2[release-pg16] or test_compute_pageserver_connection_stress2[release-pg17] or test_compute_pageserver_connection_stress2[debug-pg17] or test_compute_pageserver_connection_stress2[release-pg17] or test_compute_pageserver_connection_stress2[release-pg17]"

Flaky tests (10)

Postgres 17

• test_compute_pageserver_connection_stress2: debug-x86-64
• test_pageserver_small_inmemory_layers[True]: debug-x86-64
• test_idle_checkpoints: debug-x86-64

Postgres 16

• test_pageserver_gc_compaction_smoke: release-arm64

Postgres 15

• test_compute_pageserver_connection_stress2: release-x86-64

Postgres 14

• test_compute_pageserver_connection_stress2: release-x86-64, release-arm64
• test_pageserver_gc_compaction_smoke: release-x86-64
• test_prefetch[4]: release-x86-64
• test_pull_timeline[True]: release-x86-64

Test coverage report is not available

_{The comment gets automatically updated with the latest test results
328408b at 2024-12-05T21:35:50.141Z :recycle:}

[vadim2404]

I'm not sure that it's the right mix. It will not create different event loops, and you will fighting for the same resource (also, taking into account GIL in Python, the thread here is useless). You can just run a group of tasks (here is an example of such way to do that)

import asyncio


async def x(i):
    print(i)


async def group():
    tasks = [x(i) for i in range(10)]
    await asyncio.gather(*tasks)


if __name__ == '__main__':
    asyncio.run(group())

[vadim2404]

even in your case you can just asyncio.run(self.run_server()), why do you need the python thread?