Unable to load model

[nitinmukesh]

I installed all dependencies and after launching WebUI I select Model checkpoints (model path).
I have downloaded t2v model on local
It starts to load model but after some time it just crashes without any error. Any solution

(C:\aitools\EasyAnimate\ea_venv) C:\aitools\EasyAnimate>python app.py
C:\aitools\EasyAnimate\ea_venv\Lib\site-packages\xformers\ops\fmha\flash.py:211: FutureWarning: `torch.library.impl_abstract` was renamed to `torch.library.register_fake`. Please use that instead; we will remove `torch.library.impl_abstract` in a future version of PyTorch.
  @torch.library.impl_abstract("xformers_flash::flash_fwd")
C:\aitools\EasyAnimate\ea_venv\Lib\site-packages\xformers\ops\fmha\flash.py:344: FutureWarning: `torch.library.impl_abstract` was renamed to `torch.library.register_fake`. Please use that instead; we will remove `torch.library.impl_abstract` in a future version of PyTorch.
  @torch.library.impl_abstract("xformers_flash::flash_bwd")
C:\aitools\EasyAnimate\ea_venv\Lib\site-packages\gradio\components\dropdown.py:231: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: none or set allow_custom_value=True.
  warnings.warn(
C:\aitools\EasyAnimate\easyanimate\ui\ui.py:810: GradioUnusedKwargWarning: You have unused kwarg parameters in Image, please remove them: {'sources': 'upload'}
  start_image = gr.Image(
C:\aitools\EasyAnimate\easyanimate\ui\ui.py:837: GradioUnusedKwargWarning: You have unused kwarg parameters in Image, please remove them: {'sources': 'upload'}
  end_image   = gr.Image(label="The image at the ending of the video (图片到视频的结束图片[非必需, Optional])", show_label=False, elem_id="i2v_end", sources="upload", type="filepath")
C:\aitools\EasyAnimate\easyanimate\ui\ui.py:841: GradioUnusedKwargWarning: You have unused kwarg parameters in Video, please remove them: {'sources': 'upload'}
  validation_video = gr.Video(
C:\aitools\EasyAnimate\easyanimate\ui\ui.py:852: GradioUnusedKwargWarning: You have unused kwarg parameters in Image, please remove them: {'sources': 'upload'}
  validation_video_mask = gr.Image(
C:\aitools\EasyAnimate\easyanimate\ui\ui.py:864: GradioUnusedKwargWarning: You have unused kwarg parameters in Video, please remove them: {'sources': 'upload'}
  control_video = gr.Video(
C:\aitools\EasyAnimate\ea_venv\Lib\site-packages\gradio\utils.py:812: UserWarning: Expected 2 arguments for function <function ui.<locals>.upload_generation_method at 0x000001CE10A0EAC0>, received 1.
  warnings.warn(
C:\aitools\EasyAnimate\ea_venv\Lib\site-packages\gradio\utils.py:816: UserWarning: Expected at least 2 arguments for function <function ui.<locals>.upload_generation_method at 0x000001CE10A0EAC0>, received 1.
  warnings.warn(
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
IMPORTANT: You are using gradio version 3.48.0, however version 4.44.1 is available, please upgrade.
--------
Update diffusion transformer
### missing keys: 0;
### unexpected keys: 0;
[] []
loaded 3D transformer's pretrained weights from C:\aitools\EasyAnimate\models\Diffusion_Transformer\EasyAnimateV5-12b-zh\transformer ...

(C:\aitools\EasyAnimate\ea_venv) C:\aitools\EasyAnimate>

Tried 11.8 and 12.1

(C:\aitools\EasyAnimate\ea_venv) C:\aitools\EasyAnimate>pip list
WARNING: Ignoring invalid distribution ~orch (C:\aitools\EasyAnimate\ea_venv\Lib\site-packages)
Package                   Version
------------------------- ------------
absl-py                   2.0.0
accelerate                1.1.1
aiofiles                  23.2.1
aiohttp                   3.9.1
aiosignal                 1.3.1
albucore                  0.0.20
albumentations            1.4.21
altair                    5.4.1
annotated-types           0.6.0
antlr4-python3-runtime    4.9.3
anyascii                  0.3.2
anyio                     4.6.2.post1
attrs                     23.1.0
audioread                 3.0.1
av                        13.1.0
Babel                     2.14.0
bangla                    0.0.2
beautifulsoup4            4.12.3
blinker                   1.7.0
blis                      0.7.11
bnnumerizer               0.0.2
bnunicodenormalizer       0.1.6
cachetools                5.3.2
catalogue                 2.0.10
certifi                   2023.11.17
cffi                      1.16.0
charset-normalizer        2.1.1
click                     8.1.7
cloudpathlib              0.16.0
colorama                  0.4.6
confection                0.1.4
contourpy                 1.2.0
coqpit                    0.0.17
cycler                    0.12.1
cymem                     2.0.8
Cython                    3.0.7
datasets                  3.1.0
dateparser                1.1.8
decord                    0.6.0
deepspeed                 0.15.0
diffusers                 0.31.0
dill                      0.3.8
docopt                    0.6.2
einops                    0.8.0
encodec                   0.1.1
eval_type_backport        0.2.0
fastapi                   0.115.4
ffmpy                     0.4.0
filelock                  3.13.1
Flask                     3.0.0
fonttools                 4.47.0
frozenlist                1.4.1
fsspec                    2023.12.2
ftfy                      6.3.1
func_timeout              4.3.5
g2pkk                     0.1.2
google-auth               2.25.2
google-auth-oauthlib      1.2.0
gradio                    3.48.0
gradio_client             0.6.1
grpcio                    1.67.1
gruut                     2.2.3
gruut-ipa                 0.13.0
gruut-lang-de             2.0.0
gruut-lang-en             2.0.0
gruut-lang-es             2.0.0
gruut-lang-fr             2.0.2
h11                       0.14.0
hangul-romanize           0.1.0
hjson                     3.1.0
httpcore                  1.0.6
httpx                     0.27.2
huggingface-hub           0.26.2
idna                      3.6
imageio                   2.36.0
imageio-ffmpeg            0.5.1
importlib_metadata        8.5.0
importlib_resources       6.4.5
inflect                   7.0.0
intel-openmp              2021.4.0
itsdangerous              2.1.2
jamo                      0.4.1
jieba                     0.42.1
Jinja2                    3.1.2
joblib                    1.3.2
jsonlines                 1.2.0
jsonschema                4.23.0
jsonschema-specifications 2024.10.1
kiwisolver                1.4.5
langcodes                 3.3.0
lazy_loader               0.4
llvmlite                  0.41.1
Markdown                  3.5.1
MarkupSafe                2.1.3
matplotlib                3.8.2
mkl                       2021.4.0
mpmath                    1.3.0
msgpack                   1.0.7
multidict                 6.0.4
multiprocess              0.70.16
murmurhash                1.0.10
narwhals                  1.13.3
networkx                  2.8.8
ninja                     1.11.1.1
nltk                      3.8.1
num2words                 0.5.13
numba                     0.58.1
numpy                     1.26.3
nvidia-ml-py              12.560.30
oauthlib                  3.2.2
omegaconf                 2.3.0
opencv-python             4.10.0.84
opencv-python-headless    4.10.0.84
orjson                    3.10.11
packaging                 24.2
pandas                    1.5.3
pillow                    10.2.0
pip                       24.2
platformdirs              4.1.0
pooch                     1.8.0
preshed                   3.0.9
protobuf                  5.28.3
psutil                    5.9.7
py-cpuinfo                9.0.0
pyarrow                   18.0.0
pyasn1                    0.5.1
pyasn1-modules            0.3.0
pycparser                 2.21
pydantic                  2.9.2
pydantic_core             2.23.4
pydub                     0.25.1
pynndescent               0.5.11
pyparsing                 3.1.1
pypinyin                  0.50.0
pysbd                     0.3.4
python-crfsuite           0.9.10
python-dateutil           2.8.2
python-multipart          0.0.17
pytz                      2023.3.post1
PyYAML                    6.0.1
referencing               0.35.1
regex                     2023.10.3
requests                  2.32.3
requests-oauthlib         1.3.1
rpds-py                   0.21.0
rsa                       4.9
safetensors               0.4.5
scikit-image              0.24.0
scikit-learn              1.3.2
scipy                     1.14.1
semantic-version          2.10.0
sentencepiece             0.2.0
setuptools                75.1.0
simsimd                   6.0.3
six                       1.16.0
smart-open                6.4.0
sniffio                   1.3.1
soundfile                 0.12.1
soupsieve                 2.6
soxr                      0.3.7
spacy                     3.7.2
spacy-legacy              3.0.12
spacy-loggers             1.0.5
srsly                     2.4.8
starlette                 0.41.2
stringzilla               3.10.9
SudachiDict-core          20230927
SudachiPy                 0.6.8
sympy                     1.13.1
tbb                       2021.11.0
tensorboard               2.18.0
tensorboard-data-server   0.7.2
thinc                     8.2.2
threadpoolctl             3.2.0
tifffile                  2024.9.20
timm                      1.0.11
tokenizers                0.20.3
tomesd                    0.1.3
torch                     2.3.1+cu118
torchaudio                2.3.1+cu118
torchdiffeq               0.2.4
torchsde                  0.2.6
torchvision               0.18.1+cu118
tqdm                      4.67.0
trainer                   0.0.36
trampoline                0.1.2
transformers              4.46.2
triton                    3.0.0
TTS                       0.22.0
typing_extensions         4.12.2
tzdata                    2023.3
tzlocal                   5.2
umap-learn                0.5.5
Unidecode                 1.3.7
urllib3                   1.26.13
uvicorn                   0.32.0
wasabi                    1.1.2
wcwidth                   0.2.13
weasel                    0.3.4
websockets                11.0.3
Werkzeug                  3.0.1
wheel                     0.44.0
xformers                  0.0.27+cu118
xxhash                    3.5.0
yarl                      1.9.4
zipp                      3.21.0

[bubbliiiing]

May I ask what is the memory capacity of your machine?

[SAT431]

i too getting same issue and i have 55gb ram and 3060 gpu

[nitinmukesh]

May I ask what is the memory capacity of your machine?

Hello @bubbliiiing

Please find the info

8GB VRAM + 8GB Shared
40GB Virtual memory (SSD)

I tried all 3 options for GPU_memory_mode
Also weight_dtype to bfloat and float

Please let me know if you want any other info

[bubbliiiing]

8GB of GPU memory should not be enough, but the failure to load the model seems to be due to insufficient memory.

[C00reNUT]

I have 3090 and 32GB RAM (and about 128GB cache on SSD) and I experience the same on Ubuntu + the script logs me out from the system, ether locally or through ssh connection - I am trying i2v model

[noflytm]

• 1, I have 32 GB Ram 2 A5000 24 GB cards and 2 A4000 16 GB cards and it is just crashing my PC when I am trying to load a model

[bubbliiiing]

Currently, you may need some swap memory. We are currently trying to develop a smaller model to meet the low memory limit (30GB)

• 1, I have 32 GB Ram 2 A5000 24 GB cards and 2 A4000 16 GB cards and it is just crashing my PC when I am trying to load a model

[wywywywy]

For what it's worth, I had to increase my WSL2 to 48GB ram (8GB swap) to load the models. 32GB wasn't enough.

[bubbliiiing]

For what it's worth, I had to increase my WSL2 to 48GB ram (8GB swap) to load the models. 32GB wasn't enough.

Thank you for your test.

[bagelbig]

• 1, I have 32 GB Ram 2 A5000 24 GB cards and 2 A4000 16 GB cards and it is just crashing my PC when I am trying to load a model

Are you able to load the model so it can use more than one GPU? Might it get stuck in just one of the GPU's VRAM (24GB perhaps) ?

[noflytm]

I appreciate and thank you for your help. I've added 64GB of RAM, and now I can load the model without issues. After loading, the model starts using GPU memory, but only 1 card is in use. Do we have any approaches to distribute the GPU load across multiple cards?

[nitinmukesh]

We are currently trying to develop a smaller model to meet the low memory limit (30GB)

Thank you, looking forward.

[mikeymcfish]

I'm having the same issue on a 1 x RTX 6000 Ada 16 vCPU 188 GB RAM Runpod. Any solutions yet?