-
Notifications
You must be signed in to change notification settings - Fork 5
Data node API
Radu Marias edited this page Sep 1, 2024
·
4 revisions
The clients will communicate with data nodes with HTTP
to upload/download files using Content-Range
header. Alternatively we could offer accessing files with a BitTorrent client, Apache Arrow Flight
with direct I/O
, zero-copy
using sendfile()
to eliminate using OS's buffer and minimizing CPU usage.
We could also offer SFTP
and FTPS
as those are also accessing the file in parallel with resumable transfer.