Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for SD 3.5 IP-Adapters #9966

Open
vladmandic opened this issue Nov 19, 2024 · 3 comments
Open

Add support for SD 3.5 IP-Adapters #9966

vladmandic opened this issue Nov 19, 2024 · 3 comments

Comments

@vladmandic
Copy link
Contributor

First IP-Adapter for SD 3.5 just released at https://huggingface.co/InstantX/SD3.5-Large-IP-Adapter
with code for the modified pipeline available in the same location.

Ask is to integrate support for SD 3.5 IP-Adapter into standard t2i/i2i/inpaint pipelines

@yiyixuxu @sayakpaul @DN6 @asomoza

@sayakpaul
Copy link
Member

CC @fabiorigano as well. Since the code is already out I think it might be nice to open this to the community and we help along the way. Maybe we give it a week and if it's not picked up we integrate ourselves. @asomoza meanwhile, maybe you could run some tests as well?

@yiyixuxu
Copy link
Collaborator

cc @haofanwang too

@guiyrt
Copy link

guiyrt commented Nov 21, 2024

I started working on it and have a first draft without modifying the original code, but takes a while to test because SD3.5 is too big for my GPU.

For this PR, is the goal to have the pipeline with no changes as a community pipeline or as an optimized pipeline in src/pipelines? I noticed that it's not inheriting IPAdapterMixin loader, for example. This is my first contribution, still learning about repo structure :)

Edit: I know issues labeled as "New pipeline/model" are regarded as most difficult in the contributor guidelines, but as there is already working code, this should be easier to pick-up (hopefully) 🤞

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants