-
-
Notifications
You must be signed in to change notification settings - Fork 343
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Extension for local generators #17
Comments
@psychemedia These are all very interesting questions! Sorry for the late reply, I've been super busy as of late.
We are in the process of integrating more deeply with LangChain. LangChain does a lot of interesting things with language models, but the most relevant feature here is that they provide a fairly comprehensive "bank" of language model providers. A model provider is any class that provides an interface to one or more models; this abstraction is necessary because many models share the same Python interface (e.g. AI21, Cohere, OpenAI). You can see the full list under LangChain also includes a self-hosted provider which can use models being run locally via the transformers library. We fully plan to expose LangChain model providers out-of-the-box so users can get started quickly with the most popular and relevant language models. This would also allow for self-hosting.
LangChain also has a JavaScript library; so maybe this will be possible in the future. However, this isn't a priority for us, because there are some interesting applications of language models that require a server runtime and access to the ContentsManager. This would expand Jupyter AI's capabilities beyond simple tasks achievable via one-shot/few-shot prompting.
New functionality to Jupyter AI is added via AI modules, which are just Python packages that automatically register new modular components. Modular components are anything extensible via AI modules, including model providers. Meaning here that if we wanted to support stable diffusion, the next step is to create a new AI module
And automagically get access to the stable diffusion model. This is already what's happening with
Not sure whether this is a question regarding the design of the backend or the design of the Git repo itself.
This is a common problem in the Jupyter ecosystem, where developers often have to undergo an incredibly time-consuming "release chain" (release an upstream dependency, then release all of the downstream dependents, and repeat this process). We want to encourage developers to adopt the monorepo convention for new projects that may involve multiple packages. |
@psychemedia Thanks again for opening this issue! Since @dlqqq 's comment in late March, we have adopted LangChain in this project. We're also working on adding support for registering LangChain chains for local usage: see #46. |
@JasonWeill Thanks... having to write local chains is a bit beyond many people though? I was thinking something more like being able to say |
Although this is an older issue, I'm closing this issue in favor of #190, where we're currently tracking work for local modules. |
I'm not sure if this a a feature request or question about whether you might expect, or are hoping to see, support for other models being submitted as PRs to this repo or as plugin modules that can be added from third party installers.
Recent days have seen powerful hints that the days of running LLMs on the desktop have arrived sooner than we might have expected ( https://simonwillison.net/2023/Mar/11/llama/ ). Image generaators are also available for local use, either bundled as an an application or as a local web service. For example,
automatic111/stable-diffusion-webui
provides quite a complex UI which suggests their might be quite a rich API available thatjupyter-ai
could hook into.So as a feature request, I guess my issue is: it would be useful to have modules for stable diffusion running locally and for one of the new "portable" LLMs running locally.
(I also wonder about jupyterlite in-browser exetnensions, eg wrapping something like https://xenova.github.io/transformers.js/ ?)
As a question about third party plugins, where eg there is a module shipped to support the stable--diffusion-web-ui shipped from the automatic111 org, how would that work?
Possibly related to this, I note that jupyterlite, which originally baked in pyolite kernel, now seems to be re-engineering so that all kernels are in their own repo, and jupyterlite is just an "empty" container. Would that be a more scaleable approach for this project?
The text was updated successfully, but these errors were encountered: