Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File-like objects support #102

Open
AivanF opened this issue Jan 19, 2021 · 5 comments
Open

File-like objects support #102

AivanF opened this issue Jan 19, 2021 · 5 comments

Comments

@AivanF
Copy link

AivanF commented Jan 19, 2021

Hi! Thanks for the lib and making the world better 😊

As I can see, for now pyedflib only works with file paths, so it isn't able to read/write in memory file-like objects, e.g BytesIO? If so, it is a huge lack!

@skjerns
Copy link
Collaborator

skjerns commented Jan 19, 2021

The last time I looked into this I ran into problems passing a FILE like object to the C extension that we are wrapping (edflib). Somehow I was not able to successfully open a buffer in Python and then pass it to the C function (see also #100 ).

Please feel free to open a PR. I think it should in principle not a huge change, however, it could be tricky to pass the buffer handle in a format that is understood in C. If you find a way to do this, let me know. My quick google/stackoverflow has also not led me to a solution that supports Python native generic IO file-like buffers.

@AivanF
Copy link
Author

AivanF commented Jan 20, 2021

Not sure that I have enough skills of Cython to dig into this problem nor time to research it... But I'd like to do it one time.

I also mentioned that the lib prints loading progress, is it possible to disable? I'd like to keep the console output clean :)

@skjerns
Copy link
Collaborator

skjerns commented Jan 21, 2021

Some functions have the verbose switch, but I have not been very consistently with that. I've added them in #103 just now for all functions and set the default to False. However for now you will have to install the master version, as there will be no release just for this.

If you need any help with the PR let me know - for now I just need an option to send a generic Python IO object to C, and make the C fputc and fprintf accept these objects. I phantom this might not be possible without major modifications :-/ I even could make a simple FILE<=>open() object work last time I tried, and that should actually be compatible

fputc(255, file);

@hahnicity
Copy link

My C skills aren't very sharp, but I might be able to write a backend in Python that supports this. Any desire to support a Python backend for parsing or do you only want to support the C backend?

@skjerns
Copy link
Collaborator

skjerns commented Apr 26, 2022

As far as I remember there is already a Python implementation of edflib edflib-Python but it's horribyl slow and not using the same API. Not sure if you get the same I/O rates with using pure Python. However, it might be an idea to port the C extension to Cython?

On the flip side the advantage was so far that we did not have to maintain the backend ourselves and could just merge bugfixes etc from edflib. The edflib code has ~7500 lines of code, so my desire to port it was quite limited so far :D

Having a backend that is equally fast and reliable in Python/Cython would be great, however, I fear that it will be more work than expected. There are quite a few corner cases that need to be covered and so far edflib was one of the few ones that sticked thoroughly to the edflib specs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants