Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parquet protocol #43

Merged
merged 17 commits into from
Sep 11, 2023
Merged

Add parquet protocol #43

merged 17 commits into from
Sep 11, 2023

Commits on Aug 29, 2023

  1. Add PyCharm to .gitignore

    @nikolaj-funartech
    nikolaj-funartech committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    82e71ed View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_parquet_protocol

    @nikolaj-funartech
    nikolaj-funartech committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    64f1a89 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. Merge branch 'main' into add_parquet_protocol

    @nikolaj-funartech
    nikolaj-funartech committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    00dd02f View commit details
    Browse the repository at this point in the history

  2. Pyarrow helpers to translate ndarrays to pyarrow.Table (to the serial… 

    …ize into parquet files)
    @nikolaj-funartech
    nikolaj-funartech committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    a1c48ef View commit details
    Browse the repository at this point in the history

  3. Helpers to de/serialize array like objects from/to parquet files

    @nikolaj-funartech
    nikolaj-funartech committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    a0f4e0c View commit details
    Browse the repository at this point in the history

  4. Some constants to deal with different file formats to save array like… 

    … objects
    @nikolaj-funartech
    nikolaj-funartech committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    edc3dc2 View commit details
    Browse the repository at this point in the history

  5. Introduce the ability to use parquet files to save ndarrays

    @nikolaj-funartech
    nikolaj-funartech committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    b88e807 View commit details
    Browse the repository at this point in the history

  6. small bug corrected (forgot to call write_file with updated args)

    @nikolaj-funartech
    nikolaj-funartech committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    323f3ac View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2023

  1. Updated write_modified method from Datapackage. See #41.

    @nikolaj-funartech
    nikolaj-funartech committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    c4640b4 View commit details
    Browse the repository at this point in the history

  2. Add basic example on how to use parquet files for a Datapackage object

    @nikolaj-funartech
    nikolaj-funartech committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    24ba6f7 View commit details
    Browse the repository at this point in the history

  3. Unit tests for the io pyarrow helpers

    @nikolaj-funartech
    nikolaj-funartech committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    3d89959 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2023

  1. Moved conftest.py to tests

    @nikolaj-funartech
    nikolaj-funartech committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    23558af View commit details
    Browse the repository at this point in the history

  2. Start to write real fixtures in the tests/fixtures directory

    @nikolaj-funartech
    nikolaj-funartech committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    46ba863 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2023

  1. Unit tests for the parquet protocol

    @nikolaj-funartech
    nikolaj-funartech committed Sep 10, 2023
    Configuration menu
    Copy the full SHA
    2a4353d View commit details
    Browse the repository at this point in the history

  2. add pyarrow as requirement

    @nikolaj-funartech
    nikolaj-funartech committed Sep 10, 2023
    Configuration menu
    Copy the full SHA
    87b3437 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. Reformatting and misc cleanups

    @cmutel
    cmutel committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    e157cf0 View commit details
    Browse the repository at this point in the history

  2. Fix tests paths

    @cmutel
    cmutel committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    be6f123 View commit details
    Browse the repository at this point in the history