You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When fine tuning pretrained segmentation model using pyannote.audio==3.1.1 on well-defined and registered custom finance dataset, it shows the following error message 'PicklingError: Can't pickle <class 'pyannote.database.registry.Finance'>: attribute lookup Finance on pyannote.database.registry failed'
#1734
Open
ZhouFang928 opened this issue
Jul 1, 2024
· 4 comments
Similar Issue for pyannote.audio 2.0 pyannote/pyannote-database#65
Stating that fix has been released in pyannote.database 4.0.3
For pyannote.audio 3.0 or above, the corresponding pyannote.database version should be above 5.0.1.
**Issue if upgrading to pyannote.audio==3.3.1
If upgrading pyannote.audio to the latest version (3.3.1), it shows the following error message. If using --user option to upgrade pyannote.audio by following the error message's instruction, the system's CUDA support will be broken. So, there might be a package limitation (or other reasons) which prevents us fine tuning segmentation model as part of fine tuning pyannote v3 model, instead, we can only fine tune hyper-parameter like clustering threshold, currently.
(vtt_pyannote_tune_v3) C:\Users\fcdfaz>pip install --upgrade pyannote.audio==3.3.1
ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\Users\fcdfaz\AppData\Local\anaconda3\envs\vtt_pyannote_tune_v3\Library\bin\tbbmalloc.dll'
Consider using the --user option or check the permissions.
Is this a bug for pyannote.audio=3.1.1? How should I fix the PicklingError issue?
May I check with you which systems the package would support? Do you have a webpage listing all the system requirements for different versions of the package?
Is pyannote.audio==3.1.1 supported on linux virtualbox (VMWare Workstation Player with Ubuntu) running on a GPU windows desktop?
In addition, may I check with you whether the pyannote.ai commercial product has Windows support version, which can do both scoring and fine tuning on Windows system?
Unfortunately, I have no easy way to check whether any of these would work in a Virtualbox.
Maybe someone who did that already in pyannote user base can chime in?
Tested versions
pyannote.audio==3.1.1
System information
Windows 10 - pyannote.audio=3.1.1
Issue description
Module Version Used:
pyannote.audio==3.1.1 (or 3.0.1)
pyannote.core==5.0.0
pyannote.database==5.1.0 (or 5.0.1)
pyannote.metrics==3.2.1
pyannote.pipeline==3.0.1
Code with Problem:
from pyannote.audio.tasks import Segmentation
from types import MethodType
from torch.optim import Adam
task = Segmentation(
dataset,
duration=pretrained_segment_model.specifications.duration,
max_num_speakers=len(pretrained_segment_model.specifications.classes),
batch_size=64,
num_workers=16,
loss="bce",
vad_loss="bce")
def configure_optimizers(self):
return Adam(self.parameters(), lr=1e-4)
pretrained_segment_model.configure_optimizers = MethodType(configure_optimizers, pretrained_segment_model)
pretrained_segment_model.task = task
pretrained_segment_model.setup(stage='fit')
from pytorch_lightning.callbacks import (
EarlyStopping,
ModelCheckpoint,
RichProgressBar,
)
from pytorch_lightning import Trainer
from lightning.fabric.fabric import Fabric
from lightning.pytorch.strategies import DDPStrategy
monitor, direction = task.val_monitor
checkpoint = ModelCheckpoint(
monitor=monitor,
mode=direction,
save_top_k=1,
every_n_epochs=1,
save_last=False,
save_weights_only=False,
filename="{epoch}",
verbose=False,
)
early_stopping = EarlyStopping(
monitor=monitor,
mode=direction,
min_delta=0.0,
patience=10,
strict=True,
verbose=False,
)
callbacks = [RichProgressBar(), checkpoint, early_stopping]
trainer = Trainer(
accelerator="gpu",
devices=1,
callbacks=callbacks,
max_epochs=20,
gradient_clip_val=0.5
)
trainer.fit(pretrained_segment_model)
Error Message:
PicklingError: Can't pickle <class 'pyannote.database.registry.Finance'>: attribute lookup Finance on pyannote.database.registry failed
Similar Issue for pyannote.audio 2.0
pyannote/pyannote-database#65
Stating that fix has been released in pyannote.database 4.0.3
For pyannote.audio 3.0 or above, the corresponding pyannote.database version should be above 5.0.1.
**Issue if upgrading to pyannote.audio==3.3.1
If upgrading pyannote.audio to the latest version (3.3.1), it shows the following error message. If using --user option to upgrade pyannote.audio by following the error message's instruction, the system's CUDA support will be broken. So, there might be a package limitation (or other reasons) which prevents us fine tuning segmentation model as part of fine tuning pyannote v3 model, instead, we can only fine tune hyper-parameter like clustering threshold, currently.
(vtt_pyannote_tune_v3) C:\Users\fcdfaz>pip install --upgrade pyannote.audio==3.3.1
ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\Users\fcdfaz\AppData\Local\anaconda3\envs\vtt_pyannote_tune_v3\Library\bin\tbbmalloc.dll'
Consider using the
--user
option or check the permissions.Is this a bug for pyannote.audio=3.1.1? How should I fix the PicklingError issue?
In addition, I cannot re-run the below tutorial successfully using pyannote.audio==3.1.1
https://github.com/pyannote/pyannote-audio/blob/develop/tutorials/training_a_model.ipynb
Thanks a lot!
Minimal reproduction example (MRE)
https://github.com/pyannote/pyannote-audio/blob/develop/tutorials/training_a_model.ipynb
The text was updated successfully, but these errors were encountered: