Skip to content

Data node API

Radu Marias edited this page Sep 1, 2024 · 4 revisions

The clients will communicate with data nodes with HTTP to upload/download files using Content-Range header. Alternatively we could offer accessing files with a BitTorrent client, Apache Arrow Flight with direct I/O, zero-copy using sendfile() to eliminate using OS's buffer and minimizing CPU usage.
We could also offer SFTP and FTPS as those are also accessing the file in parallel with resumable transfer.