Allow forwarding CLI options

[jcrist]

Some options on the distributed CLI should be settable via dask-yarn (e.g. --dashboard-address). There are a few ways we could accomplish this:

• Update distributed so that all parameters are also part of the configuration. Users could then provide them via env (or via configuration Schedulers and workers not using personal config #42).
• Add an option to specify these parameters as part of the command

From jcrist/skein#176.

[mrocklin]

I think that it would be useful to start thinking about a uniform way to specify things across all of the projects.

…

On Mon, Jun 17, 2019 at 4:56 PM Jim Crist ***@***.***> wrote: Some options on the distributed CLI should be settable via dask-yarn (e.g. --dashboard-address). There are a few ways we could accomplish this: - Update distributed so that all parameters are also part of the configuration. Users could then provide them via env (or via configuration #42 <#42>). - Add an option to specify these parameters as part of the command From jcrist/skein#176 <jcrist/skein#176>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#69?email_source=notifications&email_token=AACKZTGS7G4W6MPPTRNJ74TP26QYVA5CNFSM4HYXPJLKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GZ5AHFQ>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACKZTBRTTTHKC7BX2WRGQDP26QYVANCNFSM4HYXPJLA> .

[jcrist]

I agree. I think a precursor to this would be making the CLI scripts in distributed more uniform in terms of argument parsing/config, and how processes are started/stopped.

Edit: misread your comment. I still think the above is important for yarn specific (since we need our own cli scripts), but common arguments for the cluster managers for specifying the worker/scheduler command would be good.