Whether to support multi-GPU (Nvidia) training

[zincnode]

HI,
at present, whether IREE supports multi-GPU (Nvidia) training (single-node multi-GPU or multi-node), I don't seem to see a clear introduction or guidance. I have seen plans Streams/Multi-Device Scheduling to support multiple devices (including GPU+GPU), but have not seen more details.

I tried running the given test script mnist_training.ipynb with precompiled iree-compiler, iree-runtime, iree-tools-tf and observed that all GPUs were initialized, and then only one GPU was really computing (high GPU Memory Usage, high GPU usage).