Merge changes #118
Merged
Merge changes #118
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
add mask making section
* Added mark_step for sdxl to run with pytorch xla. Also updated README with instructions for xla * adding soft dependency on torch_xla * fix some styling --------- Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: Dhruv Nair <[email protected]>
…5331) * use loralinear instead of depecrecated lora attn procs. * fix parameters() * fix saving * add back support for add kv proj. * fix: param accumul,ation. * propagate the changes.
* improvement: add missing typehints and docs to diffusers/models/attention.py * chore: convert doc strings to raw python strings add missing typehints * improvement: add missing typehints and docs to diffusers/models/adapter.py * improvement: add missing typehints and docs to diffusers/models/lora.py * docs: include suggestion by @sayakpaul in src/diffusers/models/adapter.py Co-authored-by: Sayak Paul <[email protected]> * docs: include suggestion by @sayakpaul in src/diffusers/models/adapter.py Co-authored-by: Sayak Paul <[email protected]> * docs: include suggestion by @sayakpaul in src/diffusers/models/adapter.py Co-authored-by: Sayak Paul <[email protected]> * docs: include suggestion by @sayakpaul in src/diffusers/models/adapter.py Co-authored-by: Sayak Paul <[email protected]> * Update src/diffusers/models/lora.py --------- Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: Patrick von Platen <[email protected]>
fix: StableDiffusionXLImg2ImgPipeline creation in sdxl tutorial
* I added a new doc string to the class. This is more flexible to understanding other developers what are doing and where it's using. * Update src/diffusers/models/unet_2d_blocks.py This changes suggest by maintener. Co-authored-by: Sayak Paul <[email protected]> * Update src/diffusers/models/unet_2d_blocks.py Add suggested text Co-authored-by: Sayak Paul <[email protected]> * Update unet_2d_blocks.py I changed the Parameter to Args text. * Update unet_2d_blocks.py proper indentation set in this file. * Update unet_2d_blocks.py a little bit of change in the act_fun argument line. * I run the black command to reformat style in the code --------- Co-authored-by: Sayak Paul <[email protected]>
minor fixes
* move xformers to dedicated runner * fix * remove ptl from test runner images
…rivatives (#5376) * add: freeu to the core sdxl pipeline. * add: freeu to video2video * add: freeu to the core SD pipelines. * add: freeu to image variation for sdxl. * add: freeu to SD ControlNet pipelines. * add: freeu to SDXL controlnet pipelines. * add: freu to t2i adapter pipelines. * make fix-copies.
* v1 * add tests and fix previous failing tests * fix CI * add tests + v1 `PeftLayerScaler` * style * add scale retrieving mechanism system * fix CI * up * up * simple approach --> not same results for some reason * fix issues * fix copies * remove unneeded method * active adapters! * fix merge conflicts * up * up * kohya - test-1 * Apply suggestions from code review Co-authored-by: Patrick von Platen <[email protected]> * fix scale * fix copies * add comment * multi adapters * fix tests * oops * v1 faster loading - in progress * Revert "v1 faster loading - in progress" This reverts commit ac925f8. * kohya same generation * fix some slow tests * peft integration features for unet lora 1. Support for Multiple ranks/alphas 2. Support for Multiple active adapters 3. Support for enabling/disabling LoRAs * fix `get_peft_kwargs` * Update loaders.py * add some tests * add unfuse tests * fix tests * up * add set adapter from sourab and tests * fix multi adapter tests * style & quality * style * remove comment * fix `adapter_name` issues * fix unet adapter name for sdxl * fix enabling/disabling adapters * fix fuse / unfuse unet * nit * fix * up * fix cpu offloading * fix another slow test * fix another offload test * add more tests * all slow tests pass * style * fix alpha pattern for unet and text encoder * Update src/diffusers/loaders.py Co-authored-by: Benjamin Bossan <[email protected]> * Update src/diffusers/models/attention.py Co-authored-by: Benjamin Bossan <[email protected]> * up * up * clarify comment * comments * change comment order * change comment order * stylr & quality * Update tests/lora/test_lora_layers_peft.py Co-authored-by: Patrick von Platen <[email protected]> * fix bugs and add tests * Update src/diffusers/models/modeling_utils.py Co-authored-by: Benjamin Bossan <[email protected]> * Update src/diffusers/models/modeling_utils.py Co-authored-by: Benjamin Bossan <[email protected]> * refactor * suggestion * add break statemebt * add compile tests * move slow tests to peft tests as I modified them * quality * refactor a bit * style * change import * style * fix CI * refactor slow tests one last time * style * oops * oops * oops * final tweak tests * Apply suggestions from code review Co-authored-by: Sayak Paul <[email protected]> * Update src/diffusers/loaders.py Co-authored-by: Sayak Paul <[email protected]> * comments * Apply suggestions from code review Co-authored-by: Sayak Paul <[email protected]> * remove comments * more comments * try * revert * add `safe_merge` tests * add comment * style, comments and run tests in fp16 * add warnings * fix doc test * replace with `adapter_weights` * add `get_active_adapters()` * expose `get_list_adapters` method * better error message * Apply suggestions from code review Co-authored-by: Steven Liu <[email protected]> * style * trigger slow lora tests * fix tests * maybe fix last test * revert * Update src/diffusers/loaders.py Co-authored-by: Benjamin Bossan <[email protected]> * Update src/diffusers/loaders.py Co-authored-by: Benjamin Bossan <[email protected]> * Update src/diffusers/loaders.py Co-authored-by: Benjamin Bossan <[email protected]> * Update src/diffusers/loaders.py Co-authored-by: Benjamin Bossan <[email protected]> * Apply suggestions from code review Co-authored-by: Patrick von Platen <[email protected]> Co-authored-by: Sayak Paul <[email protected]> * move `MIN_PEFT_VERSION` * Apply suggestions from code review Co-authored-by: Patrick von Platen <[email protected]> * let's not use class variable * fix few nits * change a bit offloading logic * check earlier * rm unneeded block * break long line * return empty list * change logic a bit and address comments * add typehint * remove parenthesis * fix * revert to fp16 in tests * add to gpu * revert to old test * style * Update src/diffusers/loaders.py Co-authored-by: Benjamin Bossan <[email protected]> * change indent * Apply suggestions from code review * Apply suggestions from code review --------- Co-authored-by: Patrick von Platen <[email protected]> Co-authored-by: Sourab Mangrulkar <[email protected]> Co-authored-by: Benjamin Bossan <[email protected]> Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: Steven Liu <[email protected]>
reflect stalebot change from huggingface/peft#1016
…recision training (#5407) * fix: unconditional generation example * fix: float in loss. * apply styling.
* initial script * formatting * prior trainer wip * add efficient_net_encoder * add CLIPTextModel * add prior ema support * optimizer * fix typo * add dataloader * prompt_embeds and image_embeds * intial training loop * fix output_dir * fix add_noise * accelerator check * make effnet_transforms dynamic * fix training loop * add validation logging * use loaded text_encoder * use PreTrainedTokenizerFast * load weigth from pickle * save_model_card * remove unused file * fix typos * save prior pipeilne in its own folder * fix imports * fix pipe_t2i * scale image_embeds * remove snr_gamma * format * initial lora prior training * log_validation and save * initial gradient working * remove save/load hooks * set set_attn_processor on prior_prior * add lora script * typos * use LoraLoaderMixin for prior pipeline * fix usage * make fix-copies * yse repo_id * write_lora_layers is a staitcmethod * use defualts * fix defaults * undo * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_prior.py Co-authored-by: Patrick von Platen <[email protected]> * Update src/diffusers/loaders.py Co-authored-by: Patrick von Platen <[email protected]> * Update src/diffusers/loaders.py Co-authored-by: Patrick von Platen <[email protected]> * Update src/diffusers/pipelines/wuerstchen/modeling_wuerstchen_prior.py * Update src/diffusers/loaders.py Co-authored-by: Patrick von Platen <[email protected]> * Update src/diffusers/loaders.py Co-authored-by: Patrick von Platen <[email protected]> * add graident checkpoint support to prior * gradient_checkpointing * formatting * Update examples/wuerstchen/text_to_image/README.md Co-authored-by: Pedro Cuenca <[email protected]> * Update examples/wuerstchen/text_to_image/README.md Co-authored-by: Pedro Cuenca <[email protected]> * Update examples/wuerstchen/text_to_image/README.md Co-authored-by: Pedro Cuenca <[email protected]> * Update examples/wuerstchen/text_to_image/README.md Co-authored-by: Pedro Cuenca <[email protected]> * Update examples/wuerstchen/text_to_image/README.md Co-authored-by: Pedro Cuenca <[email protected]> * Update examples/wuerstchen/text_to_image/train_text_to_image_lora_prior.py Co-authored-by: Pedro Cuenca <[email protected]> * Update src/diffusers/loaders.py Co-authored-by: Pedro Cuenca <[email protected]> * Update examples/wuerstchen/text_to_image/train_text_to_image_prior.py Co-authored-by: Pedro Cuenca <[email protected]> * use default unet and text_encoder * fix test --------- Co-authored-by: Patrick von Platen <[email protected]> Co-authored-by: Pedro Cuenca <[email protected]>
* add docs on peft diffusers integration/ Co-authored-by: Younes Belkada <[email protected]> Co-authored-by: pacman100 <[email protected]> * update URLs. * Apply suggestions from code review Co-authored-by: Younes Belkada <[email protected]> * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Steven Liu <[email protected]> * minor changes * Update docs/source/en/tutorials/using_peft_for_inference.md Co-authored-by: Steven Liu <[email protected]> * reflect the latest changes. * note about update. --------- Co-authored-by: Younes Belkada <[email protected]> Co-authored-by: pacman100 <[email protected]> Co-authored-by: Steven Liu <[email protected]>
* chore: fix typos * Update src/diffusers/pipelines/shap_e/renderer.py Co-authored-by: psychedelicious <[email protected]> --------- Co-authored-by: psychedelicious <[email protected]>
* update training examples to use HFAPI. * update training example. * reflect the changes in the korean version too. * Empty-Commit
* Add ability to mix usage of T2I-Adapter(s) and ControlNet(s). Previously, UNet2DConditional implemnetation onloy allowed use of one or the other. Adds new forward() arg down_intrablock_additional_residuals specifically for T2I-Adapters. If down_intrablock_addtional_residuals is not used, maintains backward compatibility with prior usage of only T2I-Adapter or ControlNet but not both * Improving forward() arg docs in src/diffusers/models/unet_2d_condition.py Co-authored-by: psychedelicious <[email protected]> * Add deprecation warning if down_block_additional_residues is used for T2I-Adapter (intrablock residuals) Co-authored-by: Patrick von Platen <[email protected]> * Oops my bad, fixing last commit. * Added import of diffusers utils.deprecate * Conform to max line length * Modifying T2I-Adapter pipelines to reflect change to UNet forward() arg for T2I-Adapter residuals. --------- Co-authored-by: psychedelicious <[email protected]> Co-authored-by: Patrick von Platen <[email protected]>
* fix: sdxl pipeline when unet is not available. * fix moe * account for text * ifx more * don't make unet optional. * Apply suggestions from code review Co-authored-by: Patrick von Platen <[email protected]> * split conditionals. * add optional components to sdxl pipeline * propagate changes to the rest of the pipelines. * add: test * add to all * fix: rest of the pipelines. * use pipeline_class variable * separate pipeline mixin * use safe_serialization * fix: test * access actual output. * add: optional test to adapter and ip2p sdxl pipeline tests/ * add optional test to controlnet sdxl. * fix tests * fix ip2p tests * fix more * fifx more. * use np output type. * fix for StableDiffusionXLMultiControlNetPipelineFastTests. * fix: SDXLOptionalComponentsTesterMixin * Apply suggestions from code review Co-authored-by: Patrick von Platen <[email protected]> * fix tests * Empty-Commit * revert previous * quality * fix: test --------- Co-authored-by: Patrick von Platen <[email protected]>
…rs of CLIPText (#5370) * changed channel parameters for UNET and VAE. Decreased hidden layers size with increased attention heads and intermediate size * changed the assertion check range * clean up --------- Co-authored-by: Dhruv Nair <[email protected]>
update base image for tests
* fix pipe fetcher * filter out community pipelines
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.