Groundingdino error

[KTXKIKI]

) Listening {"listenAddress": ":8082"}
2023-10-12 14:23:52 23.10.12 06:23:52.658 processor.http (D) Creating worker pool {"num": 1}
2023-10-12 14:23:52 23.10.12 06:23:52.658 sor.http.w0.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-ckjp1u200s0kejd449fg.sock"}
2023-10-12 14:23:52 23.10.12 06:23:52.659 sor.http.w0.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-ckjp1u200s0kejd449g0.sock"}
2023-10-12 14:23:52 23.10.12 06:23:52.659 sor.http.w0.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
2023-10-12 14:23:52 23.10.12 06:23:52.665 sor.http.w0.python.logger (D) Using Python handler {"handler": "main:handler"}
2023-10-12 14:23:52 23.10.12 06:23:52.666 sor.http.w0.python.logger (D) Using Python executable {"path": "/usr/bin/python3"}
2023-10-12 14:23:52 23.10.12 06:23:52.666 sor.http.w0.python.logger (D) Setting PYTHONPATH {"value": "PYTHONPATH=/opt/nuclio"}
2023-10-12 14:23:52 23.10.12 06:23:52.666 sor.http.w0.python.logger (D) Running wrapper {"command": "/usr/bin/python3 -u /opt/nuclio/_nuclio_wrapper.py --handler main:handler --event-socket-path /tmp/nuclio-rpc-ckjp1u200s0kejd449fg.sock --control-socket-path /tmp/nuclio-rpc-ckjp1u200s0kejd449g0.sock --platform-kind local --namespace nuclio --worker-id 0 --trigger-kind http --trigger-name myHttpTrigger --decode-event-strings"}
2023-10-12 14:23:38 /usr/local/lib/python3.10/dist-packages/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
2023-10-12 14:23:38 warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")
2023-10-12 14:23:38 Traceback (most recent call last):
2023-10-12 14:23:38 File "/opt/nuclio/model_handler.py", line 23, in load_network
2023-10-12 14:23:38 self.model = load_model(model_config_path=model_config_path, model_checkpoint_path=model_checkpoint_path,
2023-10-12 14:23:38 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/inference.py", line 30, in load_model
2023-10-12 14:23:38 args = SLConfig.fromfile(model_config_path)
2023-10-12 14:23:38 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/slconfig.py", line 185, in fromfile
2023-10-12 14:23:38 cfg_dict, cfg_text = SLConfig._file2dict(filename)
2023-10-12 14:23:38 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/slconfig.py", line 90, in _file2dict
2023-10-12 14:23:38 mod = import_module(temp_module_name)
2023-10-12 14:23:38 File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
2023-10-12 14:23:38 return _bootstrap._gcd_import(name[level:], package, level)
2023-10-12 14:23:38 File "", line 1050, in _gcd_import
2023-10-12 14:23:38 File "", line 1027, in _find_and_load
2023-10-12 14:23:38 File "", line 1006, in _find_and_load_unlocked
2023-10-12 14:23:38 File "", line 688, in _load_unlocked
2023-10-12 14:23:38 File "", line 883, in exec_module
2023-10-12 14:23:38 File "", line 241, in _call_with_frames_removed
2023-10-12 14:23:38 File "/tmp/tmp8eh_wv58/tmpjrzsznt8.py", line 1, in
2023-10-12 14:23:38 NameError: name 'false' is not defined. Did you mean: 'False'?
2023-10-12 14:23:38
2023-10-12 14:23:38 During handling of the above exception, another exception occurred:
2023-10-12 14:23:38
2023-10-12 14:23:38 Traceback (most recent call last):
2023-10-12 14:23:38 File "/opt/nuclio/_nuclio_wrapper.py", line 480, in
2023-10-12 14:23:38 run_wrapper()
2023-10-12 14:23:38 File "/opt/nuclio/_nuclio_wrapper.py", line 468, in run_wrapper
2023-10-12 14:23:38 loop.run_until_complete(wrapper_instance.initialize())
2023-10-12 14:23:38 File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
2023-10-12 14:23:38 return future.result()
2023-10-12 14:23:38 File "/opt/nuclio/_nuclio_wrapper.py", line 165, in initialize
2023-10-12 14:23:38 await self._initialize_context()
2023-10-12 14:23:38 File "/opt/nuclio/_nuclio_wrapper.py", line 188, in _initialize_context
2023-10-12 14:23:38 init_context_result = getattr(self._entrypoint_module, 'init_context')(self._context)
2023-10-12 14:23:38 File "/opt/nuclio/main.py", line 21, in init_context
2023-10-12 14:23:38 model = ModelHandler(labels)
2023-10-12 14:23:38 File "/opt/nuclio/model_handler.py", line 14, in init
2023-10-12 14:23:38 self.load_network(model_config_path="GroundingDINO_SwinT_OGC.py", model_checkpoint_path="groundingdino_swint_ogc.pth")
2023-10-12 14:23:38 File "/opt/nuclio/model_handler.py", line 28, in load_network
2023-10-12 14:23:38 raise Exception(f"Cannot load model {model_checkpoint_path}: {e}")
2023-10-12 14:23:38 Exception: Cannot load model groundingdino_swint_ogc.pth: name 'false' is not defined
2023-10-12 14:23:38 Exception ignored in: <function _TemporaryFileCloser.del at 0x7f1ddd299750>
2023-10-12 14:23:38 Traceback (most recent call last):
2023-10-12 14:23:38 File "/usr/lib/python3.10/tempfile.py", line 589, in del
2023-10-12 14:23:38 File "/usr/lib/python3.10/tempfile.py", line 585, in close
2023-10-12 14:23:38 FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp8eh_wv58/tmpjrzsznt8.py'
2023-10-12 14:23:39 panic: Wrapper process for worker 0 exited unexpectedly with: exit status 1
2023-10-12 14:23:39
2023-10-12 14:23:39 goroutine 41 [running]:
2023-10-12 14:23:39 github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc00039c0e0)
2023-10-12 14:23:39 /home/runner/work/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:639 +0x454
2023-10-12 14:23:39 created by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper
2023-10-12 14:23:39 /home/runner/work/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:296 +0x1fd
2023-10-12 14:23:48 /usr/local/lib/python3.10/dist-packages/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
2023-10-12 14:23:48 warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")
2023-10-12 14:23:48 Traceback (most recent call last):
2023-10-12 14:23:48 File "/opt/nuclio/model_handler.py", line 23, in load_network
2023-10-12 14:23:48 self.model = load_model(model_config_path=model_config_path, model_checkpoint_path=model_checkpoint_path,
2023-10-12 14:23:48 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/inference.py", line 30, in load_model
2023-10-12 14:23:48 args = SLConfig.fromfile(model_config_path)
2023-10-12 14:23:48 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/slconfig.py", line 185, in fromfile
2023-10-12 14:23:48 cfg_dict, cfg_text = SLConfig._file2dict(filename)
2023-10-12 14:23:48 File "/usr/local/lib/python3.10/dist-packages/groundingdino/util/slconfig.py", line 90, in _file2dict
2023-10-12 14:23:48 mod = import_module(temp_module_name)
2023-10-12 14:23:48 File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
2023-10-12 14:23:48 return _bootstrap._gcd_import(name[level:], package, level)
2023-10-12 14:23:48 File "", line 1050, in _gcd_import
2023-10-12 14:23:48 File "", line 1027, in _find_and_load
2023-10-12 14:23:48 File "", line 1006, in _find_and_load_unlocked
2023-10-12 14:23:48 File "", line 688, in _load_unlocked
2023-10-12 14:23:48 File "", line 883, in exec_module
2023-10-12 14:23:48 File "", line 241, in _call_with_frames_removed
2023-10-12 14:23:48 File "/tmp/tmpuhmaw98f/tmpa5mm56vl.py", line 1, in
2023-10-12 14:23:48 NameError: name 'false' is not defined. Did you mean: 'False'?
2023-10-12 14:23:48
2023-10-12 14:23:48 During handling of the above exception, another exception occurred:
2023-10-12 14:23:48
2023-10-12 14:23:48 Traceback (most recent call last):
2023-10-12 14:23:48 File "/opt/nuclio/_nuclio_wrapper.py", line 480, in
2023-10-12 14:23:48 run_wrapper()
2023-10-12 14:23:48 File "/opt/nuclio/_nuclio_wrapper.py", line 468, in run_wrapper
2023-10-12 14:23:48 loop.run_until_complete(wrapper_instance.initialize())
2023-10-12 14:23:48 File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
2023-10-12 14:23:48 return future.result()
2023-10-12 14:23:48 File "/opt/nuclio/_nuclio_wrapper.py", line 165, in initialize
2023-10-12 14:23:48 await self._initialize_context()
2023-10-12 14:23:48 File "/opt/nuclio/_nuclio_wrapper.py", line 188, in _initialize_context
2023-10-12 14:23:48 init_context_result = getattr(self._entrypoint_module, 'init_context')(self._context)
2023-10-12 14:23:48 File "/opt/nuclio/main.py", line 21, in init_context
2023-10-12 14:23:48 model = ModelHandler(labels)
2023-10-12 14:23:48 File "/opt/nuclio/model_handler.py", line 14, in init
2023-10-12 14:23:48 self.load_network(model_config_path="GroundingDINO_SwinT_OGC.py", model_checkpoint_path="groundingdino_swint_ogc.pth")
2023-10-12 14:23:48 File "/opt/nuclio/model_handler.py", line 28, in load_network
2023-10-12 14:23:48 raise Exception(f"Cannot load model {model_checkpoint_path}: {e}")
2023-10-12 14:23:48 Exception: Cannot load model groundingdino_swint_ogc.pth: name 'false' is not defined
2023-10-12 14:23:48 Exception ignored in: <function _TemporaryFileCloser.del at 0x7f58d9e49750>
2023-10-12 14:23:48 Traceback (most recent call last):
2023-10-12 14:23:48 File "/usr/lib/python3.10/tempfile.py", line 589, in del
2023-10-12 14:23:48 File "/usr/lib/python3.10/tempfile.py", line 585, in close
2023-10-12 14:23:48 FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpuhmaw98f/tmpa5mm56vl.py'
2023-10-12 14:23:49 panic: Wrapper process for worker 0 exited unexpectedly with: exit status 1
2023-10-12 14:23:49
2023-10-12 14:23:49 goroutine 48 [running]:
2023-10-12 14:23:49 github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).watchWrapperProcess(0xc00039c0e0)
2023-10-12 14:23:49 /home/runner/work/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:639 +0x454
2023-10-12 14:23:49 created by github.com/nuclio/nuclio/pkg/processor/runtime/rpc.(*AbstractRuntime).startWrapper
2023-10-12 14:23:49 /home/runner/work/nuclio/nuclio/pkg/processor/runtime/rpc/abstract.go:296 +0x1fd

[KTXKIKI]

metadata:
name: pth-idearesearch-dino-vit-h-cpu
namespace: cvat
annotations:
name: Grounding-Dino-CPU # 自定义万物目标检测名称
type: detector
framework: pytorch
spec: |
[
{ "id": 0, "name": "person" },
{ "id": 1, "name": "bicycle" },
{ "id": 2, "name": "car" },
{ "id": 3, "name": "motorbike" },
{ "id": 4, "name": "aeroplane" },
{ "id": 5, "name": "bus" },
{ "id": 6, "name": "train" },
{ "id": 7, "name": "truck" },
{ "id": 8, "name": "boat" },
{ "id": 9, "name": "traffic light" },
{ "id": 10, "name": "fire hydrant" },
{ "id": 11, "name": "stop sign" },
{ "id": 12, "name": "parking meter" },
{ "id": 13, "name": "bench" },
{ "id": 14, "name": "bird" },
{ "id": 15, "name": "cat" },
{ "id": 16, "name": "dog" },
{ "id": 17, "name": "horse" },
{ "id": 18, "name": "sheep" },
{ "id": 19, "name": "cow" },
{ "id": 20, "name": "elephant" },
{ "id": 21, "name": "bear" },
{ "id": 22, "name": "zebra" },
{ "id": 23, "name": "giraffe" },
{ "id": 24, "name": "backpack" },
{ "id": 25, "name": "umbrella" },
{ "id": 26, "name": "handbag" },
{ "id": 27, "name": "tie" },
{ "id": 28, "name": "suitcase" },
{ "id": 29, "name": "frisbee" },
{ "id": 30, "name": "skis" },
{ "id": 31, "name": "snowboard" },
{ "id": 32, "name": "sports ball" },
{ "id": 33, "name": "kite" },
{ "id": 34, "name": "baseball bat" },
{ "id": 35, "name": "baseball glove" },
{ "id": 36, "name": "skateboard" },
{ "id": 37, "name": "surfboard" },
{ "id": 38, "name": "tennis racket" },
{ "id": 39, "name": "bottle" },
{ "id": 40, "name": "wine glass" },
{ "id": 41, "name": "cup" },
{ "id": 42, "name": "fork" },
{ "id": 43, "name": "knife" },
{ "id": 44, "name": "spoon" },
{ "id": 45, "name": "bowl" },
{ "id": 46, "name": "banana" },
{ "id": 47, "name": "apple" },
{ "id": 48, "name": "sandwich" },
{ "id": 49, "name": "orange" },
{ "id": 50, "name": "broccoli" },
{ "id": 51, "name": "carrot" },
{ "id": 52, "name": "hot dog" },
{ "id": 53, "name": "pizza" },
{ "id": 54, "name": "donut" },
{ "id": 55, "name": "cake" },
{ "id": 56, "name": "chair" },
{ "id": 57, "name": "sofa" },
{ "id": 58, "name": "pottedplant" },
{ "id": 59, "name": "bed" },
{ "id": 60, "name": "diningtable" },
{ "id": 61, "name": "toilet" },
{ "id": 62, "name": "tvmonitor" },
{ "id": 63, "name": "laptop" },
{ "id": 64, "name": "mouse" },
{ "id": 65, "name": "remote" },
{ "id": 66, "name": "keyboard" },
{ "id": 67, "name": "cell phone" },
{ "id": 68, "name": "microwave" },
{ "id": 69, "name": "oven" },
{ "id": 70, "name": "toaster" },
{ "id": 71, "name": "sink" },
{ "id": 72, "name": "refrigerator" },
{ "id": 73, "name": "book" },
{ "id": 74, "name": "clock" },
{ "id": 75, "name": "vase" },
{ "id": 76, "name": "scissors" },
{ "id": 77, "name": "teddy bear" },
{ "id": 78, "name": "hair drier" },
{ "id": 79, "name": "toothbrush" }
]

spec:
description: Grounding dino via pytorch
runtime: 'python:3.8'
handler: main:handler
eventTimeout: 10000s
build:
image: cvat.pth.idearesearch.grounding_dino.vit_h-cpu
baseImage: ubuntu:22.04

directives:
  preCopy:
  # disable interactive frontend
    - kind: ENV
      value: DEBIAN_FRONTEND=noninteractive
  # set workdir
    - kind: WORKDIR
      value: /opt/nuclio/grounding_dino
  # install basic deps
    - kind: RUN
      value: apt-get update && apt-get -y install curl git python3 python3-pip ffmpeg libsm6 libxext6
  # install grounding-dino deps
    - kind: RUN
      value: pip3 install torch torchvision torchaudio opencv-python pycocotools matplotlib
  # install grounding-dino code
    - kind: RUN
      value: pip3 install git+https://github.com/IDEA-Research/GroundingDINO.git
  # download grounding-dino weights
    - kind: RUN
      value: curl -O https://github.com/IDEA-Research/GroundingDINO/blob/main/groundingdino/config/GroundingDINO_SwinT_OGC.py
    - kind: RUN
      value: curl -O https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
  # map pip3 and python3 to pip and python
    - kind: RUN
      value: ln -s /usr/bin/pip3 /usr/local/bin/pip && ln -s /usr/bin/python3 /usr/bin/python

triggers:
myHttpTrigger:
maxWorkers: 1
kind: 'http'
workerAvailabilityTimeoutMilliseconds: 100000000
attributes:
maxRequestBodySize: 53687091200 # 50GB

platform:
attributes:
restartPolicy:
name: always
maximumRetryCount: 3
mountMode: volume
network: cvat-develop_cvat

[KTXKIKI]

import base64
import io
import json

import yaml
from model_handler import ModelHandler
from PIL import Image

def init_context(context):
context.logger.info("Init context... 0%")

# Read labels
with open("/opt/nuclio/function.yaml", 'rb') as function_file:
    functionconfig = yaml.safe_load(function_file)

labels_spec = functionconfig['metadata']['annotations']['spec']
labels = {item['id']: item['name'] for item in json.loads(labels_spec)}

# Read the DL model
model = ModelHandler(labels)
context.user_data.model = model

context.logger.info("Init context...100%")

def handler(context, event):
context.logger.info("Run grounding dino model")
data = event.body
buf = io.BytesIO(base64.b64decode(data["image"]))
threshold = float(data.get("threshold", 0.35))
image = Image.open(buf)

results = context.user_data.model.infer(image, threshold)

return context.Response(body=json.dumps(results), headers={},
    content_type='application/json', status_code=200)

[KTXKIKI]

mogul grounding dino There is a problem with the deployment of the CPU and GPU. Is there a version update？