Only keep one, older inactive job per publication

[jpmckinney]

I forget if this was in the original requirements, but we don't need to keep more than one, to allow reversion.

Can probably do this after if job_complete:

Ensure the job succeeded (no failed tasks).

[jpmckinney]

Once jobs are deleted:

To check jsonl.gz files created in last 90 days (can change to 30 or whatever) without corresponding CSV files:

find -mtime -90 -name '.jsonl.gz' -not -name '*' | sed s/.jsonl.gz/.csv.tar.gz/ | xargs -I sh -c 'test ! -f "_" && echo _'

And to get only the unique jobs:

find -mtime -90 -name '.jsonl.gz' -not -name '*' | sed s/.jsonl.gz/.csv.tar.gz/ | xargs -I sh -c 'test ! -f "_" && echo _' | grep -Eo '/[0-9]+/' | sort -u

Can then use the output to re-run the flattener job for cases where CSVs are missing.

[jpmckinney]

Soft-blocked by #29 and #30

[jpmckinney]

Superseded by #373