Is the default learning rate a bit slow for music?

[Taikakim]

I have a dataset of somewhere around 150 of my own tracks I started training with using the default settings. Loss started from around 0.7 and even after eight hours of so training with a batch size of 38 on an A100 it only got down to around 0.66. This very different from my experience with image models where the loss with diffusion training generally hovers around 0.1 pretty much constantly.

Is it so that the original model just does not have that much music in (my style is kind of psychedelic trance/synth stuff), and there isn't just a lot to relate the training data with?

I'm now going to increase the LR by an order of magnitude just to see what happens. But it would also be interesting to see a full listing of the training data, or mainly, what kind of metadata was passed to see how I should rename my own files to place them close to positions where there is a lot of relevant data in the latent space. Or is that how this should work?

[Taikakim]

Okayyy, I made an unwrapped checkpoint at step 1200, batch size 38, and started a new train on that with a ten times higher learning rate of 5e-4, and the LR started climbing and the model to break down, so next maybe need to try something a bit more conservative: