@Docs can't index sites anymore

[drrros]

Before submitting your bug report

• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS:Rhel 9.5
- Continue version: 0.0.83
- IDE version: PyCharm 2024.3.1
- Model: Qwen2.5-Coder-32B as main model, jina-embeddings-v2-base-code-q8_0.gguf as embeddings model
- config.json:
  
{
  "models": [
    {
      "title": "Qwen2.5-30b",
      "model": "qwen2",
      "contextLength": 32768,
      "apiBase": "http://192.168.0.81:5001",
      "provider": "llama.cpp",
      "apiKey": "qwe",
      "template": "chatml",
      "completionOptions": {
        "maxTokens": 8192
      }
    }
  ],
  "customCommands": [
    {
      "name": "test",
      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    },
    {
      "name": "review",
      "prompt": "{{{ input }}}\n\nYou are a professional code reviewer. Code review the diff with relation to the existing code. Do not repeat existing code, offer suggestions and optimizations only for code changed in diff. If need, ask for more code.",
      "description": "Code review."
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5-Coder-3B",
    "contextLength": 16384,
    "apiBase": "http://192.168.0.81:5002/v1/",
    "provider": "openai",
    "model": "Qwen2.5-Coder-3B",
    "apiKey": "qwe",
    "template": "chatml"
  },
  "tabAutocompleteOptions": {
    "template": "<|fim_prefix|>{{{ prefix }}}<|fim_suffix|>{{{ suffix }}}<|fim_middle|>",
    "multilineCompletions": "auto",
    "maxPromptTokens": 2048
  },
  "contextProviders": [
    {
      "name": "diff",
      "params": {}
    },
    {
      "name": "os"
    },
    {
      "name": "docs"
    },
    {
      "name": "repo-map"
    },
    {
      "name": "url",
      "params": {}
    },
    {
      "name": "code",
      "params": {}
    },
    {
      "name": "terminal",
      "params": {}
    },
    {
      "name": "folder",
      "params": {}
    },
    {
      "name": "codebase",
      "params": {
        "nRetrieve": 50,
        "nFinal": 15,
        "useReranking": true
      }
    },
    {
      "name": "docs",
      "params": {
        "nRetrieve": 25,
        "nFinal": 5,
        "useReranking": true
      }
    }
  ],
  "slashCommands": [
    {
      "name": "edit",
      "description": "Edit selected code"
    },
    {
      "name": "comment",
      "description": "Write comments for the selected code"
    },
    {
      "name": "share",
      "description": "Export the current chat session to markdown"
    },
    {
      "name": "commit",
      "description": "Generate a git commit message"
    },
    {
      "name": "cmd",
      "description": "Generate a shell command"
    }
  ],
  "embeddingsProvider": {
    "model": "jina-bert-v2",
    "contextLength": 32768,
    "apiBase": "http://192.168.0.81:5003/v1/",
    "provider": "openai",
    "apiKey": "qwe"
  },
  "reranker": {
    "name": "free-trial"
  },
  "docs": [
    {
      "startUrl": "https://docs.djangoproject.com",
      "title": "Django",
      "faviconUrl": ""
    }
  ]
}

Description

With the recent update to "Continue version: 0.0.83" continue can't index docs anymore, this is logs:
[2024-12-15T13:32:58] Creating embeddings for 199 articles [2024-12-15T13:33:03] Creating embeddings for 199 articles [2024-12-15T13:33:23] Creating embeddings for 199 articles [2024-12-15T13:37:04] Adding 12862 embeddings to db [2024-12-15T13:37:14] Failed to fetch favicon for https://docs.djangoproject.com: TypeError: fetch failed [2024-12-15T13:37:14] Failed to find favicon for https://docs.djangoproject.com [2024-12-15T13:37:15] Adding 12862 embeddings to db [2024-12-15T13:37:25] Failed to fetch favicon for https://docs.djangoproject.com: TypeError: fetch failed [2024-12-15T13:37:25] Failed to find favicon for https://docs.djangoproject.com [2024-12-15T13:37:27] Error indexing docs Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: docs.startUrl [2024-12-15T13:37:41] Adding 12862 embeddings to db [2024-12-15T13:37:51] Failed to fetch favicon for https://docs.djangoproject.com: TypeError: fetch failed [2024-12-15T13:37:51] Failed to find favicon for https://docs.djangoproject.com [2024-12-15T13:37:52] Error indexing docs Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: docs.startUrl
happens all the time, several times tried to delete ~/.continue/index folder and restarting IDE - no success. In the embeddings model logs i see it creates many embedding, but seems to fail to write them to sqlite db.

To reproduce

No response

Log output

No response

[RomneyDa]

@drrros thanks for the report. Is it happening with any besides https://docs.djangoproject.com?