torch.OutOfMemoryError on EC2 p3.8xlarge with HunyuanVideo Preprocessing Command

[kmlFaouzi]

Description:
I'm encountering a torch.OutOfMemoryError while running a preprocessing command for the HunyuanVideo model on an AWS EC2 instance.

Environment:

• Instance Type: p3.8xlarge
• Operating System: Linux/UNIX (Ubuntu)
• GPUs: 4 x NVIDIA Tesla V100
• GPU Memory: 16 GiB per GPU

Model Details:

• Model: HunyuanVideo
• Requirements:
  • Settings (height/width/frame): 720px x 1280px x 129f
  • GPU Peak Memory: 60GB (as stated in the README)

Steps to Reproduce:

1. Use the pretrained models.
2. Run the preprocessing command:

   python hyvideo/utils/preprocess_text_encoder_tokenizer_utils.py --input_dir ckpts/llava-llama-3-8b-v1_1-transformers --
   output_dir ckpts/text_encoder

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1004.00 MiB. GPU 0 has a total capacity of 15.77 GiB of which 756.69 MiB is free. Including non-PyTorch memory, this process has 15.02 GiB memory in use. Of the allocated memory 14.58 GiB is allocated by PyTorch, and 144.06 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) --

[jash101]

You need a minimum of 45GB VRAM.
Maybe this fork can help: https://github.com/deepbeepmeep/HunyuanVideoGP

[kmlFaouzi]

of course i am using
GPUs: 4 x NVIDIA Tesla V100
GPU Memory: 16 GiB per GPU
which is 64 VRAM

[kmlFaouzi]

i tried with this link: https://github.com/deepbeepmeep/HunyuanVideoGP
and I got the same problem