Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Train model error #227

Open
mgauffin opened this issue Jul 15, 2024 · 2 comments
Open

Train model error #227

mgauffin opened this issue Jul 15, 2024 · 2 comments

Comments

@mgauffin
Copy link

I get this error when i press train model:

write filelist done
use gpus: 0
runtime\python.exe train_nsf_sim_cache_sid_load_pretrain.py -e twofer -sr 40k -f0 1 -bs 3 -g 0 -te 20 -se 5 -pg pretrained_v2/f0G40k.pth -pd pretrained_v2/f0D40k.pth -l 1 -c 0 -sw 1 -v v2 -li 11
INFO:twofer:{'train': {'log_interval': 11, 'seed': 1234, 'epochs': 20000, 'learning_rate': 0.0001, 'betas': [0.8, 0.99], 'eps': 1e-09, 'batch_size': 3, 'fp16_run': True, 'lr_decay': 0.999875, 'segment_size': 12800, 'init_lr_ratio': 1, 'warmup_epochs': 0, 'c_mel': 45, 'c_kl': 1.0}, 'data': {'max_wav_value': 32768.0, 'sampling_rate': 40000, 'filter_length': 2048, 'hop_length': 400, 'win_length': 2048, 'n_mel_channels': 125, 'mel_fmin': 0.0, 'mel_fmax': None, 'training_files': './logs\twofer/filelist.txt'}, 'model': {'inter_channels': 192, 'hidden_channels': 192, 'filter_channels': 768, 'n_heads': 2, 'n_layers': 6, 'kernel_size': 3, 'p_dropout': 0, 'resblock': '1', 'resblock_kernel_sizes': [3, 7, 11], 'resblock_dilation_sizes': [[1, 3, 5], [1, 3, 5], [1, 3, 5]], 'upsample_rates': [10, 10, 2, 2], 'upsample_initial_channel': 512, 'upsample_kernel_sizes': [16, 16, 4, 4], 'use_spectral_norm': False, 'gin_channels': 256, 'spk_embed_dim': 109}, 'model_dir': './logs\twofer', 'experiment_dir': './logs\twofer', 'save_every_epoch': 5, 'name': 'twofer', 'total_epoch': 20, 'pretrainG': 'pretrained_v2/f0G40k.pth', 'pretrainD': 'pretrained_v2/f0D40k.pth', 'version': 'v2', 'gpus': '0', 'sample_rate': '40k', 'if_f0': 1, 'if_latest': 1, 'save_every_weights': '1', 'if_cache_data_in_gpu': 0}
INFO:torch.distributed.distributed_c10d:Added key: store_based_barrier_key:1 to store for rank: 0
INFO:torch.distributed.distributed_c10d:Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 1 nodes.
gin_channels: 256 self.spk_embed_dim: 109
INFO:twofer:loaded pretrained pretrained_v2/f0G40k.pth

INFO:twofer:loaded pretrained pretrained_v2/f0D40k.pth
Process Process-1:
Traceback (most recent call last):
File "D:\RVC\Mangio\train_nsf_sim_cache_sid_load_pretrain.py", line 181, in run
utils.latest_checkpoint_path(hps.model_dir, "D_*.pth"), net_d, optim_d
File "D:\RVC\Mangio\train\utils.py", line 206, in latest_checkpoint_path
x = f_list[-1]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "multiprocessing\process.py", line 315, in _bootstrap
File "multiprocessing\process.py", line 108, in run
File "D:\RVC\Mangio\train_nsf_sim_cache_sid_load_pretrain.py", line 208, in run
net_d.module.load_state_dict(
File "D:\RVC\Mangio\runtime\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MultiPeriodDiscriminatorV2:
Missing key(s) in state_dict: "discriminators.7.convs.0.bias", "discriminators.7.convs.0.weight_g", "discriminators.7.convs.0.weight_v", "discriminators.7.convs.1.bias", "discriminators.7.convs.1.weight_g", "discriminators.7.convs.1.weight_v", "discriminators.7.convs.2.bias", "discriminators.7.convs.2.weight_g", "discriminators.7.convs.2.weight_v", "discriminators.7.convs.3.bias", "discriminators.7.convs.3.weight_g", "discriminators.7.convs.3.weight_v", "discriminators.7.convs.4.bias", "discriminators.7.convs.4.weight_g", "discriminators.7.convs.4.weight_v", "discriminators.7.conv_post.bias", "discriminators.7.conv_post.weight_g", "discriminators.7.conv_post.weight_v", "discriminators.8.convs.0.bias", "discriminators.8.convs.0.weight_g", "discriminators.8.convs.0.weight_v", "discriminators.8.convs.1.bias", "discriminators.8.convs.1.weight_g", "discriminators.8.convs.1.weight_v", "discriminators.8.convs.2.bias", "discriminators.8.convs.2.weight_g", "discriminators.8.convs.2.weight_v", "discriminators.8.convs.3.bias", "discriminators.8.convs.3.weight_g", "discriminators.8.convs.3.weight_v", "discriminators.8.convs.4.bias", "discriminators.8.convs.4.weight_g", "discriminators.8.convs.4.weight_v", "discriminators.8.conv_post.bias", "discriminators.8.conv_post.weight_g", "discriminators.8.conv_post.weight_v".

@kalbright3275
Copy link

That's odd, because each time I click on that button, this was all I received:

write filelist done
use gpus: 0
runtime\python.exe train_nsf_sim_cache_sid_load_pretrain.py -e twofer -sr 40k -f0 1 -bs 3 -g 0 -te 20 -se 5 -pg pretrained_v2/f0G40k.pth -pd pretrained_v2/f0D40k.pth -l 1 -c 0 -sw 1 -v v2 -li 11

Output information: 训练结束, 您可查看控制台训练日志或实验文件夹下的train.log

I didn't even receive a training log in my experiment logs folder or a PTH file in my weights folder...

@ZenonWrites
Copy link

That's odd, because each time I click on that button, this was all I received:

write filelist done
use gpus: 0
runtime\python.exe train_nsf_sim_cache_sid_load_pretrain.py -e twofer -sr 40k -f0 1 -bs 3 -g 0 -te 20 -se 5 -pg pretrained_v2/f0G40k.pth -pd pretrained_v2/f0D40k.pth -l 1 -c 0 -sw 1 -v v2 -li 11

Output information: 训练结束, 您可查看控制台训练日志或实验文件夹下的train.log

I didn't even receive a training log in my experiment logs folder or a PTH file in my weights folder...

you found any fix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants
@ZenonWrites @mgauffin @kalbright3275 and others