I still don't understand why increasing the batch size makes the training slower

[PGCRT]

If using a batch size of 1, (for a 512x512 training), I got 1it/s
If i'm using a batch size of 4, I got 4.3s/it and still have plenty of VRAM (80% usage)

Do I need to divide the training steps count by the batch size value or somthing?

[kijai]

That's the idea yes, probably need to adjust LR too. Iteration speed is supposed to be slower, but learning should happen faster.

[Wolchenok57]

When you use batches, you combine all images into groups of maximum 4, if the batch size is 4 (the images must be of the same size for batch training.). A batch is not a single image; it is a group of 4 (or fewer) images. In my testing, I found that a batch size other than 1 results in poor quality of output (in my case, jumbled floating limbs instead of character). You can increase the batch size if you want to stuff your VRAM with something, but I recommend letting the VRAM be free, with better results.