Is there a way to train with multi GPUs

[RaySteve312]

Is there a way to train with multi GPUs？ I do some research these days, find out maybe only 2 tools enable me to use multi-GPU training. by the way, installing dependencies constantly to use these tools feels so bad. I wonder is there a way in FluxTrainer to train with Multi GPUs? like using args or something.If we can do it in FluxTrainer, the compatibility will attract more users.

[Reginald-L]

same question

[kijai]

I don't have multiple GPUs so I never got into trying that, it's possible in the original kohya code, but the way it works is bit problematic in ComfyUI, in fact I had to remove lot of that code as it kept launching whole of ComfyUI multiple times. There's probably a proper way to do it, but it's not something I'm personally currently interested in with the nodes.

[PGCRT]

You probably want to use the Kohya GUI for these specific needs

https://github.com/kohya-ss/sd-scripts/tree/sd3