User guide to the available LLMs
This page explains how to configure Newelle for the best assistant experience based on your needs.
By default, Newelle will attempt to use several free, publicly available AI APIs (like GPT-3 providers) that don't require an API key. However, these providers may not offer the best performance or stability.
Recommended Providers:
For a more reliable and personalized experience, we strongly recommend using:
- A local model: Ideal for maximum privacy and performance on powerful hardware with dedicated GPUs. Go to Local Models section
- A provider with an API key: Offers greater control and potentially better performance than free public APIs.
Note: Multiple providers give free API Keys that have limits compatible for personal use. Here is the list of the supported providers ordered by how reccomended they are.
Groq offers a free API key with generous limits, ideal for personal use.
- Visit the Groq console: Groq Console
- Sign up with your email address (verification required).
- Navigate to the "API Keys" section and click "Create API Key."
- Give your key a name and click "Add."
- Copy the generated key.
Go in the API Key section, click on "Create API Key" and then give any name to the key. After that, click on add and copy the key that was given to you.
- Open Newelle settings.
- Choose "Groq" as the provider.
- Paste your Groq API key into the "API Key" field.
- Select the desired model.
Groq supports many of the most powerful open source AI models. You can check the list here.
At the moment vision support is only supported without system prompts and only one image a time.
Local models are the best option for maximum privacy and performance on powerful hardware with dedicated GPUs. However, they are not recommended for laptops without a dedicated GPU or if you frequently use your device unplugged. Additionally, currently available local models might offer slightly lower quality results compared to cloud-based models.
- Open Newelle settings.
- Choose "Local Model" as the provider.
- Download a compatible model from the available options.
- Select the downloaded model.
Every model has a small description that explains the model characteristics. At the moment, using other models from a file is not supported. You can track the related issue here
Google Gemini's free tier allows access to their models with limits suitable for personal use.
- Log in to Google AI Studio with your Google account: Google AI Studio
- Click "Get API Key" on the top left corner.
- Open Newelle settings.
- Important: Click the download button to install the additional modules required for using Gemini (the download is very small).
- Choose "Google Gemini API" as the provider.
- Paste your Gemini API key into the "API Key" field.
- Select the desired model.
You can see the limits for each model here. You can check the available regions for the free tier here.
Using the Open AI APIs you can not only use OpenAI API, but also any service or program that supports interference API.
Note: When using interference APIs, be sure that something is set in the API Key setting, otherwise it will give you "Connection error" Some example of programs/services that support interference APIs:
To use Groq in Newelle, go in Newelle settings, choose "OpenAI API" as a provider, and paste the key in the "API Key" setting.
From here you can also adjust multiple settings, for example:
- API Endpoint: this must be modified if you want to use and interference API. For example, if you want to use OpenRouter you can put
https://openrouter.ai/api/v1/. By default OpenAI endpoint is set - OpenAI Model: you can choose any model to put in the model parameter of the API
Newelle allows you to use a custom command run on your system to get the output message. The output of the command will be shown as a response.
This is an example that just gives you the list of prompts as a response.
Currently there are two ways to use local models in Newelle.
Newelle provides a simple way to download and use local models without any other setup.
You can easily see some information about the model, its size and download it on the fly.
While this method is easy and convenient, at the moment it has these issues:
- It has some issues loading chat history from previous conversations
- Performance might not be the best
In Newelle you can use your own Ollama instance.
- Download Ollama, the instructions are here. Note that many distros already package it in their repos.
- Install a model, you can find the list of models here. For example, to install llama3.1:8b:
ollama pull llama3.1:8b
- Start the Ollama instance:
ollama serve
You can easily use the ollama instance by selecting it in Newelle settings.
You can specify the model you want to use, and the endpoint of the instance. (If you followed the tutorial above, leave the default option)