iam unable to load model

[SAT431]

i have 55gb ram and 3060gpu

[bubbliiiing]

3060 may require sequence CPU offloading, 55GB should be sufficient

[C00reNUT]

i have 55gb ram and 3060gpu

the same goes for 3090 ... strange... the predict_i2v.py is extremely slow when loading the model into gpu and when I run 'watch nvidia-smi' i cannot even see the model loading inside memory, I haven't experience something like this before...

Is this normal behavior? the files nor the model aren't that big...

[bubbliiiing]

i have 55gb ram and 3060gpu

the same goes for 3090 ... strange... the predict_i2v.py is extremely slow when loading the model into gpu and when I run 'watch nvidia-smi' i cannot even see the model loading inside memory, I haven't experience something like this before...

Is this normal behavior? the files nor the model aren't that big...

Did you use qfloat8 before?

[SAT431]

Yes. apart from Everything it even not loading model right

i have 55gb ram and 3060gpu

the same goes for 3090 ... strange... the predict_i2v.py is extremely slow when loading the model into gpu and when I run 'watch nvidia-smi' i cannot even see the model loading inside memory, I haven't experience something like this before...
Is this normal behavior? the files nor the model aren't that big...

Did you use qfloat8 before?

Yes. apart from Everything it even not loading model right

[C00reNUT]

i have 55gb ram and 3060gpu

the same goes for 3090 ... strange... the predict_i2v.py is extremely slow when loading the model into gpu and when I run 'watch nvidia-smi' i cannot even see the model loading inside memory, I haven't experience something like this before...
Is this normal behavior? the files nor the model aren't that big...

Did you use qfloat8 before?

No, but I was running the code with GPU_memory_mode = "model_cpu_offload" not with model_cpu_offload_and_qfloat8 - I don't know whether you are using qfloat8 in model_cpu_offload, however gonna try model_cpu_offload_and_qfloat8

[bubbliiiing]

model_cpu_offload_and_qfloat8

Sorry, the 3090 needs to use model_cpu_offload_and_qfloat8 due to insufficient VRAM.

[bubbliiiing]

We are currently trying to develop a smaller model to meet the low memory limit (30GB)

[bubbliiiing]

30GB RAM will be support in #154