Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add parquet protocol #43

Merged
merged 17 commits into from
Sep 11, 2023
Merged

Add parquet protocol #43

merged 17 commits into from
Sep 11, 2023

Conversation

nikolaj-funartech
Copy link
Collaborator

Implementation of the parquet protocol for Datapackage objects. This PR essentially implements #26 but also most of the issues in the milestone Implement parquet protocol.

@nikolaj-funartech
Add PyCharm to .gitignore
82e71ed
@nikolaj-funartech
Merge branch 'main' into add_parquet_protocol
64f1a89
@nikolaj-funartech
Merge branch 'main' into add_parquet_protocol
00dd02f
@nikolaj-funartech
Pyarrow helpers to translate ndarrays to pyarrow.Table (to the serial…
a1c48ef 
…ize into parquet files)
@nikolaj-funartech
Helpers to de/serialize array like objects from/to parquet files
a0f4e0c
@nikolaj-funartech
Some constants to deal with different file formats to save array like…
edc3dc2 
… objects
@nikolaj-funartech
Introduce the ability to use parquet files to save ndarrays
b88e807
@nikolaj-funartech
small bug corrected (forgot to call write_file with updated args)
323f3ac
@nikolaj-funartech
Updated write_modified method from Datapackage. See #41.
c4640b4
@nikolaj-funartech
Add basic example on how to use parquet files for a Datapackage object
24ba6f7
@nikolaj-funartech
Unit tests for the io pyarrow helpers
3d89959
@nikolaj-funartech
Moved conftest.py to tests
23558af
@nikolaj-funartech
Start to write real fixtures in the tests/fixtures directory
46ba863
@nikolaj-funartech
Unit tests for the parquet protocol
2a4353d
@cmutel cmutel merged commit 3f6b7ff into main Sep 11, 2023
@cmutel cmutel deleted the add_parquet_protocol branch September 11, 2023 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@cmutel cmutel

Labels
None yet
Projects
None yet
Milestone
Implement parquet protocol
Development

Successfully merging this pull request may close these issues.
2 participants
@nikolaj-funartech @cmutel