Skip to content
This repository has been archived by the owner on Jan 6, 2023. It is now read-only.

Support remote cache folders that contain more than 1000 objects #8

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

struijenID
Copy link

S3's ListObject API is a paged API that returns up to 1000 entries per entry. Cache folders that contain more than 1000 objects therefore are never downloaded entirely, leading to error messages:

retrieved 1001 files from cache s3://path/to/s3/cache/<hashkey>/
Unexpected error:
[Error: ENOENT: no such file or directory, open '/path/to/git/repo/node_modules/.cache/nx/<hashkey>/terminalOutput'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/path/to/git/repo/node_modules/.cache/nx/<hashkey>/terminalOutput'
}

This commit lists all objects in the given remote cache "directory" and adds them to the list of files to be downloaded.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant