Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't seem to work with h5py #82

Open
CMCDragonkai opened this issue Nov 26, 2020 · 0 comments
Open

Doesn't seem to work with h5py #82

CMCDragonkai opened this issue Nov 26, 2020 · 0 comments

Comments

@CMCDragonkai
Copy link

I have a file I'm opening up with fs-s3fs.

This file works since I've downloaded and written to the disk.

But if I try to open up the stream in h5py, it just fails:

with fs.open_fs('s3://...').open('...', mode="rb") as weights_file:
    with h5py.File(weights_file, mode="r") as weights_hdf5:
        print(weights_hdf5)

The error is:

Traceback (most recent call last):
  File "/nix/store/cpzs1hpwzs23c41haa4dap0zjfx6xych-python3-3.7.9/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/nix/store/cpzs1hpwzs23c41haa4dap0zjfx6xych-python3-3.7.9/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/cmcdragonkai/Projects/formbay/image-classifier/image_classifier_server/services/infer.py", line 53, in __call__
    with h5py.File(weights_file, mode="r") as weights_hdf5:
  File "/nix/store/x87nsr4hhvns002c67yg43nbaknd71pn-python3.7-h5py-2.10.0/lib/python3.7/site-packages/h5py/_hl/files.py", line 408, in __init__
    swmr=swmr)
  File "/nix/store/x87nsr4hhvns002c67yg43nbaknd71pn-python3.7-h5py-2.10.0/lib/python3.7/site-packages/h5py/_hl/files.py", line 173, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 88, in h5py.h5f.open
  File "h5py/defs.pyx", line 666, in h5py.defs.H5Fopen
  File "h5py/h5fd.pyx", line 158, in h5py.h5fd.H5FD_fileobj_read
  File "/nix/store/jws4ma2692dadgbv52l2aa3s8l9mkp8s-python3.7-fs-2.4.11/lib/python3.7/site-packages/fs/iotools.py", line 116, in readinto
    return self._f.readinto(b)
  File "/nix/store/ldqzyr3f6l8rvxq5qprfak820kh7dm7x-python3.7-fs-s3fs-1.1.1/lib/python3.7/site-packages/fs_s3fs/_s3fs.py", line 154, in readinto
    return self._f.readinto()
TypeError: readinto() takes exactly one argument (0 given)

There's something wrong with the readinto() method that h5py appears to be using.

But if I just do weights_file.read() all is fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant