Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for pickling MedRecords #258

Open
JabobKrauskopf opened this issue Nov 25, 2024 · 0 comments
Open

Add support for pickling MedRecords #258

JabobKrauskopf opened this issue Nov 25, 2024 · 0 comments
Assignees
Labels
feat Related to new feature or enhancement p-backlog Priority: Backlog python Related to the python medmodels rust Related to rust medmodels

Comments

@JabobKrauskopf
Copy link
Member

JabobKrauskopf commented Nov 25, 2024

When using

train_loader = DataLoader(
            train_dataset, batch_size=self.batch_size, shuffle=True, num_workers=2
        )

and a MedRecord is inside the train_dataset, this error pops up:

======================================================================
ERROR: test_train_real_gru (main.TestRealGRU)
Test the train_real_gru method of RealGRU.

Traceback (most recent call last):
File "/Users/martin.iniguez/Desktop/projects/medmodels-plus/medmodels/data_synthesis/mtgan/model/real_gru/tests/test_real_gru.py", line 284, in test_train_real_gru
loss = self.real_gru.train_real_gru(self.train_dataset)
File "/Users/martin.iniguez/Desktop/projects/medmodels-plus/medmodels/data_synthesis/mtgan/model/real_gru/real_gru.py", line 139, in train_real_gru
for (
File "/Users/martin.iniguez/Desktop/projects/medmodels-plus/.venv/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 440, in iter
return self._get_iterator()
File "/Users/martin.iniguez/Desktop/projects/medmodels-plus/.venv/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 388, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "/Users/martin.iniguez/Desktop/projects/medmodels-plus/.venv/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1038, in init
w.start()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'builtins.PyMedRecord' object


It is solved the moment the num_workerks is equal to default value (0).

@JabobKrauskopf JabobKrauskopf added rust Related to rust medmodels python Related to the python medmodels p-backlog Priority: Backlog feat Related to new feature or enhancement labels Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat Related to new feature or enhancement p-backlog Priority: Backlog python Related to the python medmodels rust Related to rust medmodels
Projects
None yet
Development

No branches or pull requests

1 participant