Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support remote private storage by consistent use of substore #442

Closed
wants to merge 45 commits into from
Closed

Support remote private storage by consistent use of substore #442

wants to merge 45 commits into from

Commits on Jan 26, 2024

  1. allow store for ZarrLocation

    @berombau
    berombau committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    435d966 View commit details
    Browse the repository at this point in the history

  2. allow BaseStore in io_shapes

    @berombau
    berombau committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    3a1917c View commit details
    Browse the repository at this point in the history

  3. create substore with fs FSStore support

    @berombau
    berombau committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    de324a4 View commit details
    Browse the repository at this point in the history

  4. Remove old code because merge scverse/anndata#1057 (review)

    @berombau
    berombau committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    5b3c2a1 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'scverse:main' into main

    @berombau
    berombau authored Jan 26, 2024
    Configuration menu
    Copy the full SHA
    a441fee View commit details
    Browse the repository at this point in the history

  6. fix table path and sdata._path

    @berombau
    berombau committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    4ad7c44 View commit details
    Browse the repository at this point in the history

  7. readd code for consolidated table

    @berombau
    berombau committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    eb3f35b View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2024

  1. Merge branch 'scverse:main' into main

    @berombau
    berombau authored Feb 28, 2024
    Configuration menu
    Copy the full SHA
    4c150c5 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. Merge branch 'scverse:main' into main

    @berombau
    berombau authored Mar 14, 2024
    Configuration menu
    Copy the full SHA
    7bb87ba View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. implemented incremental io; tests missing

    @LucaMarconato
    LucaMarconato committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    5c73c84 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. added draft for write_metadata(); need to write new tests

    @LucaMarconato
    LucaMarconato committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    b534f6e View commit details
    Browse the repository at this point in the history

  2. wip better explanation

    @LucaMarconato
    LucaMarconato committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    e7b9433 View commit details
    Browse the repository at this point in the history

  3. fixed bug wrong order of points columns after spatial query

    @LucaMarconato
    LucaMarconato committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    ebe9f4f View commit details
    Browse the repository at this point in the history

  4. testing the copying of metadata and their inclusion in assert_element… 

    …s_are_identical
    @LucaMarconato
    LucaMarconato committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    f5f1098 View commit details
    Browse the repository at this point in the history

  5. improved readwrite tests

    @LucaMarconato
    LucaMarconato committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    695a1d8 View commit details
    Browse the repository at this point in the history

  6. added tests for incremental io

    @LucaMarconato
    LucaMarconato committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    a0974ab View commit details
    Browse the repository at this point in the history

  7. implemented write_metadata for transformations

    @LucaMarconato
    LucaMarconato committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    b429cc9 View commit details
    Browse the repository at this point in the history

  8. tests for incremental io of transformation, with separate validation for 

    writing metadata incrementally
    @LucaMarconato
    LucaMarconato committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    d5be580 View commit details
    Browse the repository at this point in the history

  9. tests for IO and incremental IO of consolidated metadata

    @LucaMarconato
    LucaMarconato committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    a9cb077 View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2024

  1. improved control over elements only on-disk/in-memory

    @LucaMarconato
    LucaMarconato committed Mar 23, 2024
    Configuration menu
    Copy the full SHA
    e13075d View commit details
    Browse the repository at this point in the history

  2. added tests for delete_element_from_disk

    @LucaMarconato
    LucaMarconato committed Mar 23, 2024
    Configuration menu
    Copy the full SHA
    fbf9e3c View commit details
    Browse the repository at this point in the history

  3. fix

    @LucaMarconato
    LucaMarconato committed Mar 23, 2024
    Configuration menu
    Copy the full SHA
    ff85b2e View commit details
    Browse the repository at this point in the history

  4. added _check_element_not_on_disk_with_different_type()

    @LucaMarconato
    LucaMarconato committed Mar 23, 2024
    Configuration menu
    Copy the full SHA
    239e0a7 View commit details
    Browse the repository at this point in the history

  5. updated changelog

    @LucaMarconato
    LucaMarconato committed Mar 23, 2024
    Configuration menu
    Copy the full SHA
    6b8069b View commit details
    Browse the repository at this point in the history

  6. fixed changelog

    @LucaMarconato
    LucaMarconato committed Mar 23, 2024
    Configuration menu
    Copy the full SHA
    1bfe022 View commit details
    Browse the repository at this point in the history

  7. attempt fix docs

    @LucaMarconato
    LucaMarconato committed Mar 23, 2024
    Configuration menu
    Copy the full SHA
    87dd1a8 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. Update src/spatialdata/_io/_utils.py 

    Co-authored-by: Kevin Yamauchi <[email protected]>
    @LucaMarconato @kevinyamauchi
    LucaMarconato and kevinyamauchi authored Mar 27, 2024
    Configuration menu
    Copy the full SHA
    7f2ec2d View commit details
    Browse the repository at this point in the history

  2. fixes from review

    @LucaMarconato
    LucaMarconato committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    bee39fc View commit details
    Browse the repository at this point in the history

  3. Merge branch 'feature/incremental_io' of https://github.com/scverse/s… 

    …patialdata into feature/incremental_io
    @LucaMarconato
    LucaMarconato committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    30a30ff View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into feature/incremental_io

    @LucaMarconato
    LucaMarconato committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    d277eea View commit details
    Browse the repository at this point in the history

  5. update test read write on disk (scverse#515) 

    * test read write on disk

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* improved tests for workarounds for incremental io

* fixed tests

* improved comment

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Luca Marconato <[email protected]>
    @ArneDefauw @pre-commit-ci @LucaMarconato
    3 people authored Mar 27, 2024
    Configuration menu
    Copy the full SHA
    57dd22e View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. Update src/spatialdata/_core/spatialdata.py 

    Co-authored-by: Giovanni Palla <[email protected]>
    @LucaMarconato @giovp
    LucaMarconato and giovp authored Mar 28, 2024
    Configuration menu
    Copy the full SHA
    582622f View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. list of names for write_element() and delete_element_from_disk()

    @LucaMarconato
    LucaMarconato committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    98037eb View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into feature/incremental_io

    @LucaMarconato
    LucaMarconato committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    5ee00c5 View commit details
    Browse the repository at this point in the history

  3. improved docs

    @LucaMarconato
    LucaMarconato committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    f2bea77 View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2024

  1. Merge branch 'main' of https://github.com/scverse/spatialdata

    @berombau
    berombau committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    1145ec6 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. Merge remote-tracking branch 'upstream/feature/incremental_io'

    @berombau
    berombau committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    f1d4f27 View commit details
    Browse the repository at this point in the history

  2. add upath support

    @berombau
    berombau committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    7d9c70b View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. move _open_zarr_store to core utils

    @berombau
    berombau committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    1e1c251 View commit details
    Browse the repository at this point in the history

  2. use _open_zarr_store instead of parse_url; use UPath instead of Path

    @berombau
    berombau committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    439d0d2 View commit details
    Browse the repository at this point in the history

  3. use zmetadata always instead of .zmetadata

    @berombau
    berombau committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    74283b1 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'scverse:main' into main

    @berombau
    berombau authored Apr 12, 2024
    Configuration menu
    Copy the full SHA
    e6b6ae9 View commit details
    Browse the repository at this point in the history

  5. add remote tests with moto

    @berombau
    berombau committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    6e04606 View commit details
    Browse the repository at this point in the history

  6. update remote tests, work on consolidated http stores

    @berombau
    berombau committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    e3e0c28 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Merge branch 'scverse:main' into main

    @berombau
    berombau authored Apr 23, 2024
    Configuration menu
    Copy the full SHA
    c120a4d View commit details
    Browse the repository at this point in the history