feat: add a persistent worker for download actions

[oxidase]

As the pip cache is disabled a persistent downloader could potentially speed up builds by removing duplicated downloads of the same wheels for different Bazel configurations.

[alexeagle]

Could it use the Bazel downloader (repository_ctx.download) so that Bazel naturally uses its --repository_cache to avoid re-fetching?

[oxidase]

Using repository_ctx.download won't work here as the wheels download is moved to a rule context where the required Python toolchain is resolved and unfortunately it is not possible to share wheels between a host and target configuration even if the resolved toolchain is the same for both configurations.

rules_pycross uses a combined way: downloads within a repository context with repository_ctx.download and builds/installs wheels within a rule context where downloaded wheels are selected from a pre-defined set of Python environments in wheel attribute . rules_python uses even a simpler approach: downloads and installs within a repository context with some fixed pre-defined Python toolchain.

Moving downloads to a repository context will require a same approach with select as in rules_pycross.

[alexeagle]

Awesome, thank you for summarizing. I think what I'm really after from this repo is your TOML parser (well, Poetry lockfile parser to be precise, there's no promise that it correctly parses arbitrary files, just the subset Poetry actually uses.)

I'd like to plug it together with pip_parse from rules_python, similar to what https://github.com/AndrewGuenther/rules_python_poetry does, but without relying on poetry being on the path during repository rule execution, and without attempting to fetch a hermetic poetry and run as a binary during repository rule execution.