Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Train issue #8

Open
rebotnix opened this issue Dec 7, 2019 · 4 comments
Open

Train issue #8

rebotnix opened this issue Dec 7, 2019 · 4 comments

Comments

@rebotnix
Copy link

rebotnix commented Dec 7, 2019

This project is really interesting.

I tried to train the model, but i always get a random list index out of range error during the training phase.

I used torch 1.2 till 1.3.1, cuda 10.1, always the same error.

Anyone has an idea how to fix that?

python3 train.py --dataset_path data/UCF-101-frames/ --split_path data/ucfTrainTestlist --num_epochs 200 --sequence_length 20 --img_dim 112 --latent_dim 512 --batch_size 64
Namespace(batch_size=64, channels=3, checkpoint_interval=5, checkpoint_model='', dataset_path='data/UCF-101-frames/', img_dim=112, latent_dim=512, num_epochs=200, sequence_length=20, split_number=1, split_path='data/ucfTrainTestlist')
cuda
--- Epoch 0 ---
[Epoch 0/200] [Batch 22/150] [Loss: 4.612639 (4.613988), Acc: 4.69% (2.31%)] ETA: 8:49:23.620145Traceback (most recent call last):
File "train.py", line 116, in
for batch_i, (X, y) in enumerate(train_dataloader):
File "/home/gary/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 801, in next
return self._process_data(data)
File "/home/gary/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
data.reraise()
File "/home/gary/.local/lib/python3.6/site-packages/torch/_utils.py", line 385, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 3.
Original Traceback (most recent call last):
File "/home/gary/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/gary/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/gary/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/opt/4tbdrive1/experiments/Action-Recognition/dataset.py", line 83, in getitem
image_paths = self._pad_to_length(image_paths)
File "/opt/4tbdrive1/experiments/Action-Recognition/dataset.py", line 67, in _pad_to_length
left_pad = sequence[0]
IndexError: list index out of range

@pcshih
Copy link

pcshih commented Jan 9, 2020

Same error as @rebotnix

@pcshih
Copy link

pcshih commented Jan 9, 2020

I do the vacant frame check

video_frame_paths = glob.glob(os.path.join(opt.dataset_frames_path, "*", "*")) for i, video_frame_path in enumerate(video_frame_paths): video_frame_len = len(glob.glob(os.path.join(video_frame_path, "*"))) if(video_frame_len==0): print(i, video_frame_path)
and extract frames again

@Soumi7
Copy link

Soumi7 commented Jul 16, 2020

@pcshih I have the same error. Which file have you made the changes to?

@key-cc
Copy link

key-cc commented Sep 6, 2022

same error, have you solved the error yet? @rebotnix @pcshih

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants