Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[community] Load Models from Sources like Civitai into Existing Pipelines #9986

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

suzukimain
Copy link
Contributor

@suzukimain suzukimain commented Nov 21, 2024

What does this PR do?

This PR adds an example that allows you to search for models from Civitai and Hugging Face by simply entering a keyword and loading models that meet the criteria into existing pipelines.

Example

!wget https://raw.githubusercontent.com/suzukimain/diffusers/refs/heads/ModelSearch/examples/model_search/search_for_civitai_and_HF.py
# Search for Civitai

from search_for_civitai_and_HF import CivitaiSearchPipeline
from diffusers import StableDiffusionPipeline


model_path = CivitaiSearchPipeline.for_civitai(
    "any",
    base_model="SD 1.5",
    download=True
)
pipe = StableDiffusionPipeline.from_single_file(model_path).to("cuda")
# Search for Hugging Face

from search_for_civitai_and_HF import HFSearchPipeline
from diffusers import StableDiffusionPipeline

model_path = HFSearchPipeline.for_HF(
           "stable",
           checkpoint_format="diffusers",
           download = False
           )

pipe = StableDiffusionPipeline.from_pretrained(model_path).to("cuda")

# or

model_path = HFSearchPipeline.for_HF(
           "stable",
           checkpoint_format="single_file",
           download = False
           )

pipe = StableDIffusionPipeline.from_single_file(model_path).to("cuda")

Arguments of HFSearchPipeline.for_HF

Name Type Default Description
search_word string The search query string.
revision string None The specific version of the model to download.
checkpoint_format string "single_file" The format of the model checkpoint.
download bool False Whether to download the model.
force_download bool False Whether to force the download if the model already exists.
include_params bool False Whether to include parameters in the returned data.
pipeline_tag string None Tag to filter models by pipeline.
hf_token string None API token for Hugging Face authentication.
skip_error bool False Whether to skip errors and return None.

Arguments of CivitaiSearchPipeline.for_civitai

Name Type Default Description
search_word string The search query string.
model_type string "Checkpoint" The type of model to search for.
base_model string None The base model to filter by.
download bool False Whether to download the model.
force_download bool False Whether to force the download if the model already exists.
civitai_token string None API token for Civitai authentication.
include_params bool False Whether to include parameters in the returned data.
skip_error bool False Whether to skip errors and return None.

search_word
Type Description
keyword Keywords to search model
url URL of either huggingface or Civitai
Local directory or file path Locally stored model paths
huggingface path The following format: < creator > / < repo >

model_type
Input Available
Checkpoint
TextualInversion
Hypernetwork
AestheticGradient
LORA
Controlnet
Poses

checkpoint_format
Argument Description
all The multifolder diffusers format checkpoint takes precedence.
single_file Only single file checkpoint are searched.
diffusers Search only for multifolder diffusers format checkpoint

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@bghira
Copy link
Contributor

bghira commented Nov 22, 2024

can it use AIR IDs to import them? thanks for this example!

edit: the fact that model API responses has the model type in it we could even do AutoPipeline.from_civitai(air=model_air) kind of thing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants