Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Reading 2kb at a time to compute the checksum limits network throughput. Bumping up to 128kb seems to give a good balance of memory usage and performance. Benchmarks done on a m5n.16xlarge EC2 instance doing a reposync on the Amazon Linux 2023 x86-64 repositories showed that this change, when combined with the (smaller) benefits of my avoiding libc IO patch, reduce system CPU time by another half second, and cut a further 3 seconds off total time: 102s (original) -> 99 (no libc buffered io) -> 95s (this patch)
- Loading branch information