integrate tokio-epoll-uring as alternative VirtualFile IO engine #5824

problame · 2023-11-08T12:53:15Z

(part of #4744 )

This PR integrates tokio-epoll-uring as an alternative IO engine for Pageserver's VirtualFile open and read operations.

The IO engine is configurable via pageserver config, and for Rust unit tests, via an env var.

The goal of integrating early is to uncover unknown problems with tokio-epoll-uring early
by exposing the code to staging & CI tests for a few days. Ideally we'd run all the CI tests for both IO engines, but, that would require twice the CI resources.

On older kernel versions, io_uring SQ and CQ is accounted as locked memory.
This PR raises the memlock ulimit wherever we execute our Rust code in CI.
See #6373 for details on memlock.
The PR that will raise memlock ulimit for staging and prod is neondatabase/aws#932;

github-actions · 2023-11-08T12:57:30Z

2367 tests run: 2270 passed, 0 failed, 97 skipped (full report)

Flaky tests (2)

Postgres 16

test_lfc_resize[tokio-epoll-uring]: debug

Postgres 15

test_crafted_wal_end[tokio-epoll-uring-last_wal_record_crossing_segment]: release

Code coverage (full report)

functions: 54.6% (10830 of 19822 functions)
lines: 81.5% (61338 of 75301 lines)

_{The comment gets automatically updated with the latest test results
7c92164 at 2024-01-25T14:43:45.850Z :recycle:}

Squashed commit of the following: commit 5ec61ce Author: Christian Schwarz <[email protected]> Date: Wed Nov 29 16:17:12 2023 +0000 bump commit 34c33d1 Author: Christian Schwarz <[email protected]> Date: Mon Nov 20 14:38:29 2023 +0000 bump commit 8fa6b76 Author: Christian Schwarz <[email protected]> Date: Mon Nov 20 11:47:19 2023 +0000 bump commit 6c359a4 Author: Christian Schwarz <[email protected]> Date: Mon Nov 20 11:33:58 2023 +0000 use neondatabase/tokio-epoll-uring#25 commit 7d484b0 Author: Christian Schwarz <[email protected]> Date: Tue Aug 29 19:13:38 2023 +0000 use WIP tokio_epoll_uring open_at for async VirtualFile::open This makes Delta/Image ::load fns fully tokio-epoll-uring commit 51b26b1 Author: Christian Schwarz <[email protected]> Date: Tue Aug 29 12:24:30 2023 +0000 use `tokio_epoll_uring` for read path commit a4e6f0c Author: Christian Schwarz <[email protected]> Date: Wed Nov 8 12:36:34 2023 +0000 Revert "revert recent VirtualFile asyncification changes (#5291)" This reverts commit ab1f37e. fixes #5479

This reverts commit cfeeda4.

.github/workflows/build_and_test.yml

pageserver/src/tenant/block_io.rs

pageserver/src/lib.rs

…afe`; addresses #5824 (comment)

pageserver/src/virtual_file.rs

pageserver/src/virtual_file/io_engine.rs

pageserver/src/virtual_file.rs

stacked atop #5824

Cargo.toml

jcsp

I'm good with this: only doubt is whether any of the VirtualFile changes might introduce subtle perf regressions, but that's what our shiny new benchmarks are for.

problame · 2024-01-19T16:27:03Z

Edit: moved to #6373 (comment)

problame · 2024-01-19T18:13:54Z

Edit: moved to #6373 (comment)

This is prep work for integrating support for runtime-configurable io engines (=> tokio-epoll-uring).

Apart from sticking closer to the comment above the function, this reduces the diff in the next patch.

…exercise it Test `test_eof_before_buffer_full` fails, as discussed in #5824 (comment)

Code is unused though, next commit hooks it up to the CI

- control via env var PAGESERVER_VIRTUAL_FILE_IO_ENGINE - if an io engine other than std-fs is used, it shows up in the test name; this is so that we can continue to use the flaky tests database - raise memlock limit & while at it also raise shmem limit for the Rust tests. It's need on our older runners that use an older 5.10.X LTS kernel, where io_uring SQ and CQ still counted towards the rlimit, see #6373 (comment) for details. Co-authored-by: Alexander Bayandin <[email protected]>

Running both combinations bloats CI times too much. So, we switch to tokio-epoll-uring for 2-3 days, so that the code gets exposure. Then we switch it back a few days before the next release so that the `std-fs` engine gets coverage, because that's what we're going to continue using in prod for the time being.

PR #5824 introduced the concept of io engines in pageserver and implement `tokio-epoll-uring` in addition to our current method, `std-fs`. We used `tokio-epoll-uring` in CI for a few days to get more exposure to the code. Now it's time to switch CI back so that we test with `std-fs` as well, because that's what we're (still) using in production.

…#6492) PR #5824 introduced the concept of io engines in pageserver and implemented `tokio-epoll-uring` in addition to our current method, `std-fs`. We used `tokio-epoll-uring` in CI for a day to get more exposure to the code. Now it's time to switch CI back so that we test with `std-fs` as well, because that's what we're (still) using in production.

stacked atop #5824

problame mentioned this pull request Nov 8, 2023

Epic: scalable async disk IO (tokio-epoll-uring) #4744

Closed

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from 6b5992d to 0003744 Compare November 8, 2023 14:55

arpad-m mentioned this pull request Nov 13, 2023

Cleanup compact_level0_phase1 fsyncing #5852

Merged

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from c1b1da2 to 6c359a4 Compare November 20, 2023 11:36

problame added a commit that referenced this pull request Nov 30, 2023

Revert "pull in WIP: integrate tokio-epoll-uring #5824"

1e5533c

This reverts commit cfeeda4.

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from 5ec61ce to f51a08e Compare December 1, 2023 13:47

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from f51a08e to 0ca94f5 Compare December 11, 2023 16:26

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from 256b417 to b388180 Compare January 12, 2024 11:20

problame changed the title ~~WIP: integrate tokio-epoll-uring~~ integrate tokio-epoll-uring Jan 12, 2024

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from b388180 to ccd2de9 Compare January 12, 2024 11:30

problame changed the title ~~integrate tokio-epoll-uring~~ integrate tokio-epoll-uring as alternative VirtualFile IO engine Jan 12, 2024

problame commented Jan 12, 2024

View reviewed changes

.github/workflows/build_and_test.yml Outdated Show resolved Hide resolved

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from ccd2de9 to ccf8ffd Compare January 12, 2024 17:06

bayandin self-requested a review January 12, 2024 18:13

bayandin force-pushed the problame/integrate-tokio-epoll-uring/wip branch from 0d1bd27 to f90c159 Compare January 15, 2024 19:03

bayandin added the run-benchmarks Indicates to the CI that benchmarks should be run for PR marked with this label label Jan 15, 2024

bayandin approved these changes Jan 15, 2024

View reviewed changes

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from 75ebc39 to e99a3a1 Compare January 17, 2024 12:41

problame marked this pull request as ready for review January 17, 2024 12:46

problame requested a review from a team as a code owner January 17, 2024 12:46

problame requested review from arpad-m, jcsp and VladLazar and removed request for a team and arpad-m January 17, 2024 12:46

problame mentioned this pull request Jan 17, 2024

tokio-epoll-uring: root-cause hung pageserver rust unit tests #6373

Closed

arpad-m reviewed Jan 17, 2024

View reviewed changes

pageserver/src/tenant/block_io.rs Outdated Show resolved Hide resolved

pageserver/src/lib.rs Show resolved Hide resolved

problame added a commit that referenced this pull request Jan 17, 2024

fixup(refactor(VirtualFile): use owned buffers for reads): avoid `uns…

f940868

…afe`; addresses #5824 (comment)

arpad-m approved these changes Jan 17, 2024

View reviewed changes

VladLazar reviewed Jan 17, 2024

View reviewed changes

pageserver/src/virtual_file.rs Show resolved Hide resolved

pageserver/src/virtual_file/io_engine.rs Show resolved Hide resolved

pageserver/src/virtual_file.rs Show resolved Hide resolved

problame added a commit that referenced this pull request Jan 17, 2024

tokio-epoll-uring integration: support more operations

a55835e

stacked atop #5824

problame mentioned this pull request Jan 17, 2024

tokio-epoll-uring: use it on the layer-creating code paths #6378

Merged

jcsp reviewed Jan 18, 2024

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

jcsp approved these changes Jan 18, 2024

View reviewed changes

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from aac2a83 to 423b4e1 Compare January 19, 2024 11:16

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from 298b3e6 to cb6a8df Compare January 23, 2024 18:29

problame added 3 commits January 24, 2024 16:06

refactor(VirtualFile): add its own OpenOptions wrapper

9b2eb09

This is prep work for integrating support for runtime-configurable io engines (=> tokio-epoll-uring).

refactor(VirtualFile): switch to OwnedFd

4da903e

noop(VirtualFile): read_exact_at: actually vendor the rust std version

17ff475

Apart from sticking closer to the comment above the function, this reduces the diff in the next patch.

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from cb6a8df to 2cf5a4c Compare January 24, 2024 16:37

problame added a commit that referenced this pull request Jan 24, 2024

refactor+test(read_exact_at): move to _impl function, add tests that …

e95db72

…exercise it Test `test_eof_before_buffer_full` fails, as discussed in #5824 (comment)

problame and others added 6 commits January 25, 2024 12:12

refactor(VirtualFile): use owned buffers for reads

7a24861

refactor(VirtualFile): configurable IO engine

b14f4dd

feature(VirtualFile): implement tokio-epoll-uring IO engine

9ea11a7

feature(test_runner): support choosing Pageserver VirtualFile IO engine

a8ab0ae

Code is unused though, next commit hooks it up to the CI

problame force-pushed the problame/integrate-tokio-epoll-uring/wip branch from a8c9912 to 7c92164 Compare January 25, 2024 13:19

problame merged commit 918b03b into main Jan 26, 2024
51 checks passed

problame deleted the problame/integrate-tokio-epoll-uring/wip branch January 26, 2024 08:25

problame mentioned this pull request Jan 26, 2024

CI: switch back to std-fs io engine for soak time before next release #6492

Merged

problame added a commit that referenced this pull request Feb 5, 2024

tokio-epoll-uring integration: support more operations

a3c88a8

stacked atop #5824

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

integrate tokio-epoll-uring as alternative VirtualFile IO engine #5824

integrate tokio-epoll-uring as alternative VirtualFile IO engine #5824

problame commented Nov 8, 2023 •

edited

Loading

github-actions bot commented Nov 8, 2023 •

edited

Loading

Postgres 16

Postgres 15

jcsp left a comment

problame commented Jan 19, 2024 •

edited

Loading

problame commented Jan 19, 2024 •

edited

Loading

integrate tokio-epoll-uring as alternative VirtualFile IO engine #5824

integrate tokio-epoll-uring as alternative VirtualFile IO engine #5824

Conversation

problame commented Nov 8, 2023 • edited Loading

github-actions bot commented Nov 8, 2023 • edited Loading

2367 tests run: 2270 passed, 0 failed, 97 skipped (full report)

Postgres 16

Postgres 15

Code coverage (full report)

jcsp left a comment

Choose a reason for hiding this comment

problame commented Jan 19, 2024 • edited Loading

problame commented Jan 19, 2024 • edited Loading

problame commented Nov 8, 2023 •

edited

Loading

github-actions bot commented Nov 8, 2023 •

edited

Loading

problame commented Jan 19, 2024 •

edited

Loading

problame commented Jan 19, 2024 •

edited

Loading