API for auditing physical files and file metadata

[stevenwinship]

What this PR does / why we need it: Find Datasets with missing files so Admins can either delete the file reference or work with authors to re-upload the files.
See: IQSS/dataverse.harvard.edu#220

Which issue(s) this PR closes:

• Closes #

Special notes for your reviewer:

Suggestions on how to test this: Create multiple Datasets with multiple files. If running in Docker locally delete a file from docker-dev-volumes/app/data/store...
call the api and see the missing file listed in the json response.
Other test could include deleting a FileMetadata row from the DB
Request specific Datasets as well as firstId and lastId

Does this PR introduce a user interface change? If mockups are available, please link/include them here: No

Is there a release notes update needed for this change?: Included

Additional documentation:

Preview docs at https://dataverse-guide--11016.org.readthedocs.build/en/11016/api/native-api.html#datafile-audit

[coveralls]

coverage: 21.825% (-0.03%) from 21.856%
when pulling 26e8574 on 220-audit-physical-files
into 61b8046 on develop.

[pdurbin]

I took a quick pass through the docs and code. @stevenwinship please let me know what you think.

[pdurbin]

Do we use this pattern of passing in the URL form of a PID minus "https://" anywhere else? It seems ok. Can we pass in the normal PIDs (the non-URL form) instead?

[stevenwinship]

9/21/22 Durbin:

Batch Exports Through the API
...
curl http://localhost:8080/api/admin/metadata/:persistentId/reExportDataset?persistentId=doi:10.5072/FK2/AAA000

[pdurbin]

No, it's different... "doi.org/10... vs. doi:10...".

In this PR we. should use the pattern from reExportDataset.

[stevenwinship]

updated the doc

[pdurbin]

Same. Easier parsing would be nice.

[stevenwinship]

re-formatted the json output:

"missingFiles": [
{
"StorageIdentifier": "s3://dvn-cloud:298910",
"label": "jihad_metadata_edited.csv"
}
]

[pdurbin]

Great! Thanks. Do we need the directoryLabel too?

[stevenwinship]

added directoryLabel

[pdurbin]

I didn't run the code but API tests are passing as of 3eec366 and the latest commits were just docs and reformatting. Approved.

[github-actions]

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:220-audit-physical-files

ghcr.io/gdcc/configbaker:220-audit-physical-files

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

[ofahimIQSS]

Merging PR - Testing Passed
Uploading Screen Recording 2024-12-02 at 4.27.37 PM.mov…