Feel free to add a PR :)

Yeah, I may do that. Before I dive in, are there any known reasons why reference only wouldn't work with the XL controlnet?

(What I'm working around is that because textual inversions aren't supported yet for XL - thank you for your help with understanding that btw - I now need a new solution for a problem that I was previously solving with TIs.)

The ControlNet accepts additional embeddings for SDXL. Maybe that needs to be rejigged.

any updates on this?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

Commenting so this isn't stale. Still following this issue. This pipeline would be really useful.

Welcome to PRs.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

PRs are welcome for this addition. But I'm just curious to understand why IP adapters wouldn't be able to provide similar functionality?

PRs are welcome for this addition. But I'm just curious to understand why IP adapters wouldn't be able to provide similar functionality?

I originally struggled to get good results with the ip_adapter, but I was using another implementation. I have since tried the current one in diffusers with load_ip_adapter and it does seem to work great and basically has the same effect as reference-only so maybe this is obsolete.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

Need this too

need this

PRs are welcome for this addition. But I'm just curious to understand why IP adapters wouldn't be able to provide similar functionality?

IP-Adapter use CLIP image encoder to produce semantic image features which may be lack of detailed texture features. While dense features from reference only controlnet will work better in specific situations.

Can someone post an example for this? I mean the reference image, the prompt and the result, so I can compare it and if indeed it's better than IP Adapters, add it to my to-do list if no one from the community wants to do it.