Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use compress.write_zip() with s3fs source #556

Open
james-emerton opened this issue Oct 26, 2022 · 0 comments
Open

Unable to use compress.write_zip() with s3fs source #556

james-emerton opened this issue Oct 26, 2022 · 0 comments

Comments

@james-emerton
Copy link

I believe this ticket should be for this package, but the issue is caused by implementation details of the fs-s3fs package. Attempting to create an archive of files from s3 using write_zip() fails with the following error:

Traceback (most recent call last):
  File "/Users/james/dev/spire/mount-s3-archive/test.py", line 7, in <module>
    write_zip(source, sys.argv[2])
  File "/Users/james/dev/spire/mount-s3-archive/.venv/lib/python3.10/site-packages/fs/compress.py", line 78, in write_zip
    mt = info.modified or datetime.utcnow()
  File "/Users/james/dev/spire/mount-s3-archive/.venv/lib/python3.10/site-packages/fs/info.py", line 322, in modified
    self._require_namespace("details")
  File "/Users/james/dev/spire/mount-s3-archive/.venv/lib/python3.10/site-packages/fs/info.py", line 128, in _require_namespace
    raise MissingInfoNamespace(namespace)
fs.errors.MissingInfoNamespace: namespace 'details' is required for this attribute

Code:

import sys

import fs
from fs.compress import write_zip

source = fs.open_fs(sys.argv[1])
write_zip(source, sys.argv[2])

Executing the above code as test.py s3://some-bucket/ test.zip will fail with the above error while a local source directory will compress nicely.

The issue seems to be that S3FS directory entries don't expose a details namespace. It's unclear if that's correct or not, but the code in write_zip() seems to assume that details is always available (but makes allowance for Info.modified to be a falsy value).

james-emerton added a commit to spiresystems/pyfilesystem2 that referenced this issue Oct 27, 2022
This will check the `stat` and `details` namespaces before defaulting to the current time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant