Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STOP training when activating tab with another Comfyui (asyncio???) #83

Open
Maelstrom2014 opened this issue Oct 16, 2024 · 0 comments
Open

Comments

@Maelstrom2014
Copy link

2024-10-17 00:18:53.043081 #171 [FluxTrainLoop]: 3061.75s
INFO Prompt executed in 13816.86 seconds main.py:138
2024-10-17 00:25:10 ERROR Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) base_events.py:1785
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "asyncio\events.py", line 84, in _run
File "asyncio\proactor_events.py", line 165, in _call_connection_lost
ConnectionResetError: [WinError 10054] Удаленный хост принудительно разорвал
существующее подключение
2024-10-17 00:25:10.220050 [LogConsole] client [11d7004667534f9cb1676528db12417a], console [0d47d0ee-b156-4cb3-b8d7-e4a6d472567c], disconnected
2024-10-17 00:25:18 INFO got prompt server.py:600
ERROR Failed to validate prompt for output 199: execution.py:822
ERROR * (prompt): execution.py:824
ERROR - Required input is missing: images execution.py:826
ERROR * easy imageSave 199: execution.py:842
ERROR - Required input is missing: images execution.py:844
ERROR Output will be ignored execution.py:846
2024-10-17 00:25:18.314709 queue_counter: 0
Epoch 7/12 - steps: 44%|█████████████████████ | 2630/6000 [3:41:57<4:44:24, 5.06s/it, avr_loss=0.374]2024-10-17 00:25:21.866731
MY Loss: 0.396391898393631, global_step: 2630

2024-10-17 00:25:21.871743
loss_total: 190.18249675631523

2024-10-17 00:25:21.871743 Will save to: global_loss_list_n7_seven_768_rank128_bf16
2024-10-17 00:25:21.873756 Will save to: global_epoch_list_n7_seven_768_rank128_bf16
2024-10-17 00:25:21.876732 ss_num_batches_per_epoch: 508
2024-10-17 00:25:21.879733 mins: [ 206 223 2134 2266 2249]
Epoch 7/12 - steps: 44%|█████████████████████ | 2631/6000 [3:42:05<4:44:22, 5.06s/it, avr_loss=0.374]2024-10-17 00:25:29.482791
MY Loss: 0.41960006952285767, global_step: 2631

2024-10-17 00:25:29.486794
loss_total: 190.26320031285286

2024-10-17 00:25:29.487794 Will save to: global_loss_list_n7_seven_768_rank128_bf16
2024-10-17 00:25:29.489801 Will save to: global_epoch_list_n7_seven_768_rank128_bf16
2024-10-17 00:25:29.490793 ss_num_batches_per_epoch: 508
2024-10-17 00:25:29.492792 mins: [ 206 223 2134 2266 2249]
Epoch 7/12 - steps: 44%|█████████████████████ | 2631/6000 [3:42:05<4:44:23, 5.06s/it, avr_loss=0.375]2024-10-17 00:25:29 INFO Processing interrupted execution.py:375

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant