Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

implement open_at #25

Merged
merged 7 commits into from
Nov 22, 2023
Merged

implement open_at #25

merged 7 commits into from
Nov 22, 2023

Conversation

problame
Copy link
Collaborator

@problame problame commented Nov 20, 2023 •
edited
Loading

(Stacked atop #24 )

  • Vendor in OpenOptions from tokio-uring.git:d5e90539bd6d1c518e848298564a098c300866bc
  • Remove the OpenOptions::open() function which is
    tokio-uring-specific
  • Remove examples because they mention tokio_uring and hence fail cargo test
  • Add the extension trait that allows for conversion of OpenOptions into
    an io_uring::squeue::Entry
  • Use it throughout the code base.

This PR obsoletes #21

@problame problame requested review from koivunej and arpad-m November 20, 2023 11:23
@problame problame mentioned this pull request Nov 20, 2023
Closed
problame added a commit to neondatabase/neon that referenced this pull request Nov 20, 2023
@problame
use neondatabase/tokio-epoll-uring#25
c1b1da2
problame added a commit to neondatabase/neon that referenced this pull request Nov 20, 2023
@problame
use neondatabase/tokio-epoll-uring#25
6c359a4
@problame problame force-pushed the problame/uring-common branch from 16d09a9 to 04cb1bd Compare November 20, 2023 11:44
@problame
vendor (and add ext trait for) IoBuf/IoBufMut from tokio-uring
716ef11 
For [`open_at`
support](#21) we
need an `OpenOptions` struct. Sadly we can't re-use the one from
`tokio-uring` because it doesn't have a public API for the conversion of
OpenOptions into the relevant libc flags.

We created [a PR asking for such an
API](neondatabase/tokio-uring#1), but, in the
meantime, let's unblock ourselves by vendoring the pieces of
`tokio-uring` that we need, and cusotmize them as needed.

This PR starts that effort by vendoring the `IoBuf`/`IoBufMut` traits.
`OpenOptions` will follow in a later PR.

The files that reproduce the `tokio-uring` LICENSE text at the top
are copied from `tokio-uring.git:d5e90539bd6d1c518e848298564a098c300866bc`.

Files without it were written by myself.
@problame
implement open_at
65dc5cf 
* Vendor in `OpenOptions` from `tokio-uring.git:d5e90539bd6d1c518e848298564a098c300866bc`
* Remove the `OpenOptions::open()` function which is
  `tokio-uring`-specific
* Remove doc comments that mention the `tokio-uring` `File` type
* Add the extension trait that allows for conversion of OpenOptions into
  an `io_uring::squeue::Entry`
* Use it throughout the code base.
@problame problame force-pushed the problame/uring-common branch from 04cb1bd to 716ef11 Compare November 20, 2023 14:29
@problame problame mentioned this pull request Nov 20, 2023
Merged
@problame problame assigned problame and koivunej and unassigned problame Nov 21, 2023
@problame problame mentioned this pull request Nov 21, 2023
Merged
arpad-m
arpad-m approved these changes Nov 21, 2023
View reviewed changes
tokio-epoll-uring/src/ops/open_at.rs Outdated Show resolved Hide resolved
tokio-epoll-uring/src/ops/open_at.rs Outdated Show resolved Hide resolved
tokio-epoll-uring/src/ops/open_at.rs Show resolved Hide resolved
Base automatically changed from problame/uring-common to main November 22, 2023 07:24
problame added a commit that referenced this pull request Nov 22, 2023
@problame
vendor IoBuf/IoBufMut from tokio-uring (#24)
91d67e2 
In the draft PR for [`open_at`
support](#21) we
need an `OpenOptions` struct. Sadly we can't re-use the one from
`tokio-uring` because it doesn't have a public API for the conversion of
OpenOptions into the relevant libc flags.

We created [a PR asking for such an
API](neondatabase/tokio-uring#1), but, in the
meantime, let's unblock ourselves by vendoring the pieces of
`tokio-uring` that we need, and cusotmize them as needed.

This PR starts that effort by vendoring the `IoBuf`/`IoBufMut` traits as
well as `tokio-uring`'s approach to support slice-like operations.
Support for `OpenOptions` will follow as part of [the PR that adds
`open_at` support on top of this
PR](#25).

The files that reproduce the `tokio-uring` LICENSE text at the top are
copied from `tokio-uring.git:d5e90539bd6d1c518e848298564a098c300866bc`.
Files without it were written by myself.

To make `cargo test` pass, I had to remove the examples in the doc
comments.
@problame problame merged commit 53c832d into main Nov 22, 2023
2 checks passed
@problame problame deleted the problame/openat_v2 branch November 22, 2023 08:43
problame added a commit to neondatabase/neon that referenced this pull request Nov 29, 2023
@problame
pull in WIP: integrate tokio-epoll-uring #5824
cfeeda4 
Squashed commit of the following:

commit 5ec61ce
Author: Christian Schwarz <[email protected]>
Date:   Wed Nov 29 16:17:12 2023 +0000

    bump

commit 34c33d1
Author: Christian Schwarz <[email protected]>
Date:   Mon Nov 20 14:38:29 2023 +0000

    bump

commit 8fa6b76
Author: Christian Schwarz <[email protected]>
Date:   Mon Nov 20 11:47:19 2023 +0000

    bump

commit 6c359a4
Author: Christian Schwarz <[email protected]>
Date:   Mon Nov 20 11:33:58 2023 +0000

    use neondatabase/tokio-epoll-uring#25

commit 7d484b0
Author: Christian Schwarz <[email protected]>
Date:   Tue Aug 29 19:13:38 2023 +0000

    use WIP tokio_epoll_uring open_at for async VirtualFile::open

    This makes Delta/Image ::load fns fully tokio-epoll-uring

commit 51b26b1
Author: Christian Schwarz <[email protected]>
Date:   Tue Aug 29 12:24:30 2023 +0000

    use `tokio_epoll_uring` for read path

commit a4e6f0c
Author: Christian Schwarz <[email protected]>
Date:   Wed Nov 8 12:36:34 2023 +0000

    Revert "revert recent VirtualFile asyncification changes (#5291)"

    This reverts commit ab1f37e.

    fixes #5479
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@koivunej koivunej koivunej left review comments

@arpad-m arpad-m arpad-m approved these changes

Assignees

@koivunej koivunej

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@problame @koivunej @arpad-m