Look into https://github.com/nelsontky/gh-pages-url-shortener / shorten URLs via data.pybamm.org subdomain

[agriyakhetarpal]

which is a neat way to shorten URLs for release artifacts

[santacodes]

Can we not bind the URLs to something like a subdomain specific for pybamm-data? or maybe a separate path within current URLs for each release?

[agriyakhetarpal]

Can we not bind the URLs to something like a subdomain specific for pybamm-data?

Hi, @santacodes, could you please elaborate? I am not sure I follow, are you suggesting something like GitHub Pages?

or maybe a separate path within current URLs for each release?

We do not have storage limits on GitHub Releases, and the files are too small to cover the per-release limit (2 GB). However, I chose to version the releases here so that we can have links between PyBaMM versions and pybamm-data – that a particular version of PyBaMM, say, v24.5, can rely on a particular release (v1.0.0) here, since the links to the files will be hardcoded in the PyBaMM source code. If a newer PyBaMM v2X.Y will need newer renditions of these files or new files, we will create a new v1.0.1 or v1.1.0 release with all of the files again plus the new/changed files, so that PyBaMM v24.5 and v2X.Y shall download different files. Also, we cannot have folders in a GitHub Release, so having separate paths will be difficult unless we upload tarballs (and we don't want to deal with unpacking tarballs when we can download the files directly)

P.S. We could have used CalVer as well instead of SemVer, but it should be fine either way since we just needed a method to version the files, so the choice of scheme is irrelevant.

[santacodes]

I meant we could maybe have a subdomain for example data.pybamm.org for hosting different releases of data files. Might just be much more organised and more centralised for data files. We could make them release-specific by changing the path like you mentioned for e.g data.pybamm.org/v1.0.0 and data.pybamm.org/v2.X.X and so on. and just redirect them to the source code links. I guess this might not be the need of the hour but might as well just put it here for future considerations.

[agriyakhetarpal]

I am also considering putting out a minimal Python package for pybamm-data on PyPI that can be added as a required dependency for PyBaMM. The package will contain no legible source code, will be pure Python, and will just the data files sorted into folders – the public API will provide simple methods or property attributes of the form

import pybamm_data
import pandas as pd

my_data1 = pybamm_data.Ecker2015
my_data2 = pybamm.COMSOL_results.comsol_1C

df1 = pd.read_csv(my_data1)
df2 = pd.read_json(my_data2)

i.e., they will be able provide Path or PosixPath objects, with extra information or custom types for identifying the file formats, and even integrating support for the SHA-256 checksums can be looked into (we will have to see how to do that, though). This will also likely be useful for pybamm-cookiecutter, and we can plan on doing this in the coming weeks as a part of GSoC if others feel this would be beneficial – because while this would shorten the links to all of the files, it might be slightly overkill as well.

xref: pybamm-team/pybamm-cookie#1

Let me open a new issue about this, though.