I found a strange issue!!!

[Lym2333]

I found a strange issue, I was using nnUNetv1
when training ending, it didn't exit the terminal or stop after the training was over? I must use crtl+c to break it!
After that, I tried to run predict program, when it comes to over, the system abruptly exits the user login screen, it happend 2 more times. I'm really confused. (Using 'all' folds)

[Lym2333]

@FabianIsensee Hi, sorry to disturb you.
Now the prediction program is working well, maybe it's an accidental situation about the exit to the login interface.(2 more times, whatever :) )
But what bothers me now is that the terminal is still running after the training is completed. It seems like that the program was stuck in the 'for' loop of training.
I'm confused, could you explain this to me?

Best
Lym

[Lym2333]

#2071
Here is the same problem.

[Lym2333]

#1885
Maybe this is solution, I will try it.

[Lym2333]

I monitored the memory usage for a training session, and the CPU and GPU memory usage was low, but the memory(ddr4) usage was not normal, and it remained the same for long time. (33GB/256GB)
Now I'm sure the problem comes from this issue, I'm running a test traning to verify #1885 's solution.
Wish me luck.