Latent-Diffusion Models

[Adamdad]

Model/Dataset/Scheduler description

Recent trends show that diffusion models, especially latent diffusion models, have been very successful for image/video generation tasks. While the current codebase supports IDDPM, I believe supporting LDM for tasks like unconditional image generation or even text-to-image generation would be very helpful for the openmmlab family.

Open source status

• The model implementation is available
• The model weights are available.

Provide useful links for the implementation

https://github.com/CompVis/latent-diffusion

[LeoXing1996]

Hey @Adamdad, thanks for your advice!

We plan to support ADM, LDM, and other diffusion models (some have already been supported but are under review 👀 ). Sampling acceleration methods such as DDIM and PNDM are in planning as well.

BTW, we have refactored MMGeneration and MMEditing based on MMEngine and MMCV 2.0, and we have recently consolidated MMGeneration into MMEditing. All the above models will be supported in MMEditing in the future.
Please feel free to contact us if you want to make some contribution 🚀 !

[zeakey]

@LeoXing1996 Did you mean you are planning to merge mmgen to mmedit? Seems like mmedit is more frequently updated than mmgen.

[LeoXing1996]

@zeakey We have already merged MMGen to MMEditing, and we are working on StableDiffusion in MMEditing. If you have any advice about the dev-1.x branch, don't hesitate to contact us.

[zeakey]

@LeoXing1996 Sure. I'm currently working on diffusion models and glad to see a mm-series package supports diffusion models. I will keep an eye on mmedit.

PS:
Just checked out mmedit and it seems like some basic diffusion models (e.g DDPM, improved DDPM) are missing?

[LeoXing1996]

@zeakey Thanks for your interest. We are working on DDPM and Improved-DDPM and will finish in early January.

[lxtGH]

Hey @Adamdad, thanks for your advice!

We plan to support ADM, LDM, and other diffusion models (some have already been supported but are under review 👀 ). Sampling acceleration methods such as DDIM and PNDM are in planning as well.

BTW, we have refactored MMGeneration and MMEditing based on MMEngine and MMCV 2.0, and we have recently consolidated MMGeneration into MMEditing. All the above models will be supported in MMEditing in the future. Please feel free to contact us if you want to make some contribution 🚀 !

Hi!! So it is better to use the MMEditing rather than this codebase for diffusion research? @LeoXing1996

[LeoXing1996]

Hey @Adamdad, thanks for your advice!
We plan to support ADM, LDM, and other diffusion models (some have already been supported but are under review 👀 ). Sampling acceleration methods such as DDIM and PNDM are in planning as well.
BTW, we have refactored MMGeneration and MMEditing based on MMEngine and MMCV 2.0, and we have recently consolidated MMGeneration into MMEditing. All the above models will be supported in MMEditing in the future. Please feel free to contact us if you want to make some contribution 🚀 !

Hi!! So it is better to use the MMEditing rather than this codebase for diffusion research? @LeoXing1996

Of course, @lxtGH!!