Koboldcpp would really benefit from RAG and Websearch

[adrian52999]

So, let me start by saying that Koboldcpp is truly the most pleasant to work with out of all the endpoints. It is simultaneously the easiest to install and work with, and it is also one of the fastest ones. The RP possibilities it opens are great, however I think that by adding RAG and web search it would branch out and possibly compete with the likes of open webui.

How likely would this be?

[Asherathe]

Why have so many people been asking about RAG and web search lately? Those are frontend features, and there are tons of LLM frontends that implement them already.

[adrian52999]

Yes, you are right. I could use Kobold as a backend in frontends like open-webui or tl/dw, and I did - it's really nice. Yet, I still feel that RAG and websearch would help push Kobold in the one-size-fits-all direction it seems to be going...

[esolithe]

Yes, you are right. I could use Kobold as a backend in frontends like open-webui or tl/dw, and I did - it's really nice. Yet, I still feel that RAG and websearch would help push Kobold in the one-size-fits-all direction it seems to be going...

I presume this would be one to ask for here then? Kobold CPP is the back end - focused specifically on the generation, whereas the front end you see in the web browser is Kobold Lite.

While it is early days, there's a "sort of" context searching feature which attempts to pull relevant snippets from your whole history (it does not use vectorising or similar, more of an enhanced text search), and if successful might be possible to expand to documents as a separate PR in the future (mostly if there's interest and the base mechanism works well enough for most people).

The link to the PR is here if you're interested to follow / try it out.