Error when excuting LineArtPreprocessor

[TruckMan19]

Error occurred when executing LineArtPreprocessor:

[Errno 2] No such file or directory: 'D:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts\lllyasviel/Annotators\cache\models--lllyasviel--Annotators\blobs\c686ced2a666b4850b4bb6ccf0748031c3eda9f822de73a34b8979970d90f0c6.incomplete'

I'm trying to run the Lineart preprocessor as part of a workflow I saw online, and it suggested using Controlnet_Aux. When it gets to the "Realistic lineart" node, it throws the error. Above.

Not entirely sure what to make of this. It looks like it's looking for a lineart model in a subfolder? Is there a way to redirect it? I have lineart controlnet models downloaded, but in my regular controlnet/models path. Just a bit confused about what to do here. if you have any guidance would really appreciate it.

[BagGuyArt]

I'm gonna be annoying and not give you an answer, so sorry.

I'm having the same exact problem. It's not just LineArtPreprocessor that's not working. Most of them are not. Canny is the only one that works, which I think comes pre-installed with Comyfui. I tried installing and uninstalling Controlnet_Aux, tried deleting the folder, tried using the install.bat, tried a fresh install of Comfyui, and tried updating it. Nothing.

[BagGuyArt]

Error occurred when executing LineArtPreprocessor:

[Errno 2] No such file or directory: 'C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts\lllyasviel/Annotators\cache\models--lllyasviel--Annotators\blobs\c686ced2a666b4850b4bb6ccf0748031c3eda9f822de73a34b8979970d90f0c6.incomplete'

File "C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\node_wrappers\lineart.py", line 19, in execute
model = LineartDetector.from_pretrained(HF_MODEL_NAME, cache_dir=annotator_ckpts_path).to(model_management.get_torch_device())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\controlnet_aux\lineart_init_.py", line 102, in from_pretrained
model_path = custom_hf_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\controlnet_aux\util.py", line 243, in custom_hf_download
model_path = hf_hub_download(repo_id=pretrained_model_or_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\huggingface_hub\file_download.py", line 1419, in hf_hub_download
with temp_file_manager() as temp_file:
File "contextlib.py", line 137, in enter
File "C:\AI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\huggingface_hub\file_download.py", line 1403, in _resumable_file_manager
with open(incomplete_path, "ab") as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Close

[TruckMan19]

I'm gonna be annoying and not give you an answer, so sorry.

I'm having the same exact problem. It's not just LineArtPreprocessor that's not working. Most of them are not. Canny is the only one that works, which I think comes pre-installed with Comyfui. I tried installing and uninstalling Controlnet_Aux, tried deleting the folder, tried using the install.bat, tried a fresh install of Comfyui, and tried updating it. Nothing.

No worries! I'm glad I'm not the only one. Yeah I realized it's all the preprocessors except "inpainting" and "canny" for whatever reason. I'm not really sure, my guess is there's some kind of file path, but I couldn't figure it out.

I've tried redownloading. I tried messing around with the file config file, but I was just guessing and it didn't lead to anything

[Fannovel16]

@TruckMan19 @BagGuyArt Try to delete comfyui_controlnet_aux folder in custom_nodes (don't remove through the manager) then reinstall it

[BagGuyArt]

@TruckMan19 @BagGuyArt Try to delete comfyui_controlnet_aux folder in custom_nodes (don't remove through the manager) then reinstall it

Same issue. I even tried reinstalling it using a git clone instead (tried reinstalling it both ways). Are there any steps post-manager-installation that we're missing? I don't want to speak for TruckMan19 but I know I'm fully capable of being and idiot and missing obvious shit.

[TruckMan19]

@TruckMan19 @BagGuyArt Try to delete comfyui_controlnet_aux folder in custom_nodes (don't remove through the manager) then reinstall it

I still get the same error unfortunately. What I realize, is when you install for the first time, the "CKPT" folders is not present in the first install. Then when you launch comfyUI for the first time, the CKPT folder is created within the "comfyui_controlnet_aux" folder.

So I would think maybe something is going wrong here? Am I supposed to download something after installation? Or should I type in a different path and get the program to look for the CKPTs in a different location?

INFO -> Using ckpts path: E:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts

On startup I get this message in the CMD. And that folder is essentially empty except for some other folders. I'm thinking the issue is here perhaps?

This might be over my head, but what is supposed to be in the CKPT folder? I had thought that's where the controlnet models themselves were supposed to reside? My controlnet models are saved and download elsewhere though.

model = LineartDetector.from_pretrained(HF_MODEL_NAME, cache_dir=annotator_ckpts_path).to(model_management.get_torch_device())
File "E:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\controlnet_aux\lineart_init_.py", line 102, in from_pretrained
model_path = custom_hf_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
File "E:\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\src\controlnet_aux\util.py", line 243, in custom_hf_download
model_path = hf_hub_download(repo_id=pretrained_model_or_path,

Also just to include some more details I get that message, it looks like it's trying to do a huggingface download maybe? "Custom_hf_download". Is that maybe the issue? Should I be downloading something from HF?

[TruckMan19]

@Fannovel16 I think I understand the problem possibly, and I was wondering if you could help out. When the extension is fully installed and properly functioning, where are the actual Checkpoints of the controlnet models located? What is the full directory?

I think the issue is right now, after you install the extension, then run the code, it's trying to download the checkpoints for the various controlnet methods but running into an error. I already have all the controlnet models manually downloaded, so can I just create a folder and paste them in there. I'm trying to follow the code but struggling a bit, what is the first script that's triggered when you try to run a preprocessor? I can try and figure out from there

[Fannovel16]

@TruckMan19

what is the first script that's triggered when you try to run a preprocessor?

It's __init__.py. But what you want to check is https://github.com/Fannovel16/comfyui_controlnet_aux/blob/main/src/controlnet_aux/util.py#L210

[TruckMan19]

Cheers. Will take a look, I know a little bit of python this is actually a great way to learn more. It's one thing to write a script, but a project like this, with multiple files interacting with each other is great. It is tough to follow at times, I need to look into how to run the code line by line and inspect the variables each step of the way.

And yes you're right, that is the line that's throwing the error around that line. I tried to add a line "print(annotator_ckpts_path)" and some of the other variables to see what path it was actually referencing but I couldn't see any output. Might need to try something else.

What exactly is it trying to download from HF? The controlnet checkpoints? I can download those manually, I just don't know what filepath they would go in. Would you be able to tell me what directory the checkpoints are supposed to end up in? I can try to just do it manually see if that fixes it.

The issue seems to be post install, it doesn't seem to know where to find the actual checkpoints for the models. Not sure why. I also tried modifying the YAML file, to point it straight to the folder containing the controlnet CKPTS but no dice. My models are also .safetensor, I noticed the code was explicitly looking for ".ckpts" so I might try that as well see if that fixes it.

[Fannovel16]

@TruckMan19 It downloads preprocessor models making the hint image for ControlNet, not the ControlNet itself. They are not the same thing. Most preprocessors are also neural networks

[TruckMan19]

@TruckMan19 It downloads preprocessor models making the hint image for ControlNet, not the ControlNet itself. They are not the same thing. Most preprocessors are also neural networks

Oh damn I'm dumb. Gotcha, that makes a lot more sense. Hm, ok.

[TruckMan19]

@TruckMan19 @BagGuyArt Try to delete comfyui_controlnet_aux folder in custom_nodes (don't remove through the manager) then reinstall it

Same issue. I even tried reinstalling it using a git clone instead (tried reinstalling it both ways). Are there any steps post-manager-installation that we're missing? I don't want to speak for TruckMan19 but I know I'm fully capable of being and idiot and missing obvious shit.

Hey not sure if you got this to work or not but I just "figured it out".

What worked for me: I just did a fresh ComfyUI install on a thumbdrive, then I manually installed (git clone) this repo and then I tried using lineart preprocessor (and a few others) and it worked, it went ahead and downloaded the right models. So then I ust copied the entire "comfyui_controlnet_aux" folder from my new install to my old install and it worked.

What I think would also work: Go to your "Annotators" folder in this file path:
ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts\lllyasviel\Annotators

In that folder maybe clear out everything. Then in that location, download the files from here: https://huggingface.co/lllyasviel/Annotators/tree/main

So at the end of the day your annotator folder should just have these "pth" and ".pt" files. These files seem to be to be the preprocessing files that the code is trying to download when it's throwing an error. So manually installing them might be a workaround.

You can try to just do one as a test if you want, looks like realistic lineart uses:"sk_model.pth" and "sk_model2.pth". So try to download those two from the link above, and then try using lineart preprocessor.

If that doesn't work, you can just do what i did. Install comfyUI fresh on a new thumbdrive so your existing one doesn't get wiped out and you can just run the preprocessors on a sample image, and then copy it over. Hope it works for you.

[TruckMan19]

@TruckMan19 It downloads preprocessor models making the hint image for ControlNet, not the ControlNet itself. They are not the same thing. Most preprocessors are also neural networks

Thank you so much for your help, really appreciate it. Finally got it working, bit of a workaround, must be something on my install that's causing a conflict.

Can I ask you a python question if you don't mind? Is it possible to try to debug this kind of stuff using "print" commands? For instance if I think the problem is with "annotator_ckpts_path", can I try to print the variable to see where the code is actually looking?

I tried using print and it didn't output the result anywhere, so wondering if there's a better way possibly of doing it.

At any rate, thank you very much! I hope to get good enough at Python one day to be able to do work similar to what you have here.

[Layer-norm]

These models in the picture are all you need in comfyui_controlnet_aux. The "annotator_ckpts_path" used to refer files from lllyasviel/Annotators at huggingface. But there are many other good preprocessor models at present. So it refers to the root directory of these models now. For instance, you can find yolo_nas_s_fp16.onnx at hr16/yolo-nas-fp16

Besides if you want to print some thing, you can use log function from log.py like this

from .log import log
annotator_ckpts_path = "./ckpts"
log.info(f"Using ckpts path: {annotator_ckpts_path}")

Hope my answer will help you.

[BagGuyArt]

@TruckMan19 Sorry for the very late response. WOrked like a charm! THanks!

Helping me to kick my A1111 habit.

[Fannovel16]

Moving to #264