Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs: make FileHandle.readableWebStream always create byte streams #55461

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

isker
Copy link
Contributor

@isker isker commented Oct 19, 2024

The original implementation of the experimental
FileHandle.readableWebStream API created non-type: 'bytes' streams, which prevented callers from creating mode: 'byob' readers from the returned stream, which means they could not achieve the associated "zero-copy" performance characteristics.

Then, #46933 added a parameter allowing callers to pass the type parameter down to the ReadableStream constructor, exposing the same semantics to callers of FileHandle.readableWebStream.

But there is no point to giving callers this choice: FileHandle-derived streams are by their very nature byte streams. We should not require callers to explicitly opt in to having byte stream semantics. Moreover, I do not see a situation in which callers would ever want to have a non-bytes stream: bytes-streams only do anything differently than normal ones if mode: 'byob' is passed to getReader.

So, remove the options parameter and always create a ReadableStream with type: 'bytes'.

Fixes #54041.

@nodejs-github-bot nodejs-github-bot added fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. labels Oct 19, 2024
doc/api/fs.md Show resolved Hide resolved
@isker isker force-pushed the filehandle-readablestream-always-bytes branch from 0e81802 to 57bfbb3 Compare October 19, 2024 23:56
Copy link

codecov bot commented Oct 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.41%. Comparing base (5e5af29) to head (41c448c).
Report is 217 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #55461      +/-   ##
==========================================
- Coverage   88.41%   88.41%   -0.01%     
==========================================
  Files         652      654       +2     
  Lines      186918   187855     +937     
  Branches    36077    36145      +68     
==========================================
+ Hits       165271   166095     +824     
- Misses      14901    15000      +99     
- Partials     6746     6760      +14     
Files with missing lines Coverage Δ
lib/internal/fs/promises.js 98.23% <100.00%> (-0.01%) ⬇️

... and 179 files with indirect coverage changes

@RedYetiDev RedYetiDev added the experimental Issues and PRs related to experimental features. label Nov 7, 2024
@isker isker force-pushed the filehandle-readablestream-always-bytes branch from 57bfbb3 to 88496b4 Compare November 7, 2024 01:32
Copy link
Member

@RedYetiDev RedYetiDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry again for the butchering of my last comment.

Here's my idea to gracefully make users aware of this change. We start by warning that this option has no effect, and then in the next semver, we remove the warning entirely.

lib/internal/fs/promises.js Outdated Show resolved Hide resolved
lib/internal/fs/promises.js Outdated Show resolved Hide resolved
@RedYetiDev
Copy link
Member

RedYetiDev commented Nov 7, 2024

Assuming this lands in 23.X.0, we can remove the warning in 23.(X + 1).0

Although, because this is experimental, a warning isn't technically required, so you have the final say (IMO) whether there is one to add

@isker isker force-pushed the filehandle-readablestream-always-bytes branch from 88496b4 to d747a6f Compare November 7, 2024 03:28
@karlhorky
Copy link
Contributor

karlhorky commented Nov 7, 2024

Edit: Sorry, it's already marked as such!

Should my issue also be added in a "Fixes" comment?

@isker
Copy link
Contributor Author

isker commented Nov 14, 2024

@RedYetiDev can you take another look?

@isker isker requested a review from RedYetiDev November 17, 2024 15:45
lib/internal/fs/promises.js Outdated Show resolved Hide resolved
lib/internal/fs/promises.js Outdated Show resolved Hide resolved
lib/internal/fs/promises.js Outdated Show resolved Hide resolved
@isker isker force-pushed the filehandle-readablestream-always-bytes branch from 946019e to 86a1021 Compare November 17, 2024 16:06
The original implementation of the experimental
`FileHandle.readableWebStream` API created non-`type: 'bytes'` streams,
which prevented callers from creating `mode: 'byob'` readers from the
returned stream, which means they could not achieve the associated
"zero-copy" performance characteristics.

Then, nodejs#46933 added a parameter allowing callers to pass the `type`
parameter down to the ReadableStream constructor, exposing the same
semantics to callers of `FileHandle.readableWebStream`.

But there is no point to giving callers this choice: FileHandle-derived
streams are by their very nature byte streams. We should not require
callers to explicitly opt in to having byte stream semantics. Moreover,
I do not see a situation in which callers would ever want to have a
non-bytes stream: bytes-streams only do anything differently than normal
ones if `mode: 'byob'` is passed to `getReader`.

So, remove the `options` parameter and always create a ReadableStream
with `type: 'bytes'`.

Fixes nodejs#54041.
@isker isker force-pushed the filehandle-readablestream-always-bytes branch from 86a1021 to 41c448c Compare November 17, 2024 17:24
@isker isker requested review from RedYetiDev and jasnell November 20, 2024 01:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
experimental Issues and PRs related to experimental features. fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

filehandle.readableWebStream() chunks return incompatible ArrayBuffer instead of Uint8Array
5 participants