Add v1 data

[forsyth2]

v1 data should also be included on e3sm_data_docs.

Currently, v1 data is linked from https://e3sm.org/data/get-e3sm-data/released-e3sm-data/. Ideally, we should move all those links to e3sm_data_docs so all the data is linked from one coherent site. Note https://acme-climate.atlassian.net/wiki/spaces/ED/pages/4495441922 provides some v1 data as well.

[forsyth2]

@chengzhuzhang @TonyB9000 Is the plan to leave the HPSS path as-is for the v1 data? E.g., /home/g/golaz/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison rather than the more uniform /home/projects/e3sm/www/ prefix?

Reasons to switch:

• It would simplify automatically extracting simulation information for E3SM Data Docs
• It would provide consistency for users
• It doesn't rely on a single user's HPSS path

Reasons to keep as-is:

• It could be burdensome/time-consuming to move all the v1 data to new directories. (Especially considering we're already on to v3 now).

[chengzhuzhang]

Another registration is our HPSS project quota, we may run out of space after all the v1 data is copied. I think use original path is fine. We should use a consistent structure for v3 for sure.

[chengzhuzhang]

@forsyth2 my assessment above about use original HPSS path is not correct, because these datasets can only be able to accessed world-wide (from https://portal.nersc.gov/archive/home/projects/e3sm/www/) if they are placed under `/home/projects/e3sm/www/.

[forsyth2]

@chengzhuzhang Oh that is a good point. So should we begin a process of moving data already on HPSS to be under that path?

[chengzhuzhang]

We can wait a little bit and work with Tony to figure out the best option to create the backup...

[TonyB9000]

@chengzhuzhang @forsyth2 Please see my latest comment at the bottom of the Confluence page

https://acme-climate.atlassian.net/wiki/spaces/IPD/pages/4503633923/2024-08-02+IG+data+publication+Continue+publication+archive+backup

Let me know if I can proceed with further archive creation. To save space, we probably want to transfer and delete as we go, or else incur a (temporary) 130+TB footprint on disk.

[forsyth2]

To save space, we probably want to transfer and delete as we go, or else incur a (temporary) 130+TB footprint on disk.

I think that is fine. It looks like many simulations on https://acme-climate.atlassian.net/wiki/spaces/ED/pages/4495441922/V1+Simulation+backfill+WIP in fact have an HPSS path and backup path already -- i.e., there already is a copy for many simulations.