cromwell_logs -- download all error/output logs in a zip file?

[tefirman]

Brief description of the problem:

• Troubleshooting/debugging a WDL script is currently a bit difficult with PROOF.
• PROOF does a good job of pointing to the location of output/error logs in rhino, but that still requires logging in to rhino.
• Might be easier to group all of those logs together in a zip file and have the user download them to their local machine.
• Could eventually be a button on PROOF, but felt like the core functionality needs to occur more upstream.

[sckott]

@tefirman Okay if I take this?

Hm, I don't think the actual logs being returned is part of the Cromwell REST API. But maybe I'm missing something. Do you know Taylor if logs can be returned?

[tefirman]

@sckott, that would be awesome if you don't mind! You're right, unfortunately Cromwell only returns the location of the logs, not the logs themselves, so this would require some manuevering, not entirely sure how though.

[sckott]

Yeah I don't know how easily we can make this happen. I'll take a look see

[sckott]

@dtenenba AFAICT Cromwell API doesn't allow one to fetch the contents of workflow log files, but only the location (path). Do you think there's an easy enough way to securely fetch the log file contents in addition to the location of said file?

It seems there is a way via configuration to send log files somewhere else so they don't get cleaned up https://cromwell.readthedocs.io/en/latest/Logging/#workflow-logs but that still leaves the problem of fetching those log files. Perhaps it'd be easier though to fetch them for the client (e.g., PROOF app or an R client) if log files were pushed into an S3 bucket for example rather than somewhere else on the cluster

[sckott]

@tefirman We discussed this with Dan yeah? Or maybe not?