LLM Servers & Models
Overview
Before your agents can generate responses, Evolvable.ai needs to know which AI service to use and which specific model to call. This is configured in two steps:
- LLM Server — a connection to an AI provider (OpenAI, Azure OpenAI, Ollama, etc.), including credentials.
- LLM Model — a specific model available on that server (e.g.
gpt-4o,mistral,nomic-embed-text), enabled for use by your agents.
Once a server is connected and models are enabled, you can select them when configuring agents and knowledge base settings.
Table of Contents
LLM Servers
Supported Providers
Evolvable.ai can connect to the following AI providers:
| Provider | Type | Notes |
|---|---|---|
| OpenAI | Cloud | GPT-4, GPT-4o, GPT-3.5, and embedding models |
| Azure OpenAI | Cloud | OpenAI models hosted on Microsoft Azure |
| Ollama | Self-hosted | Run open-source models on your own infrastructure |
| Cloud | Google AI models (coming soon) | |
| Anthropic | Cloud | Claude models (coming soon) |
Adding a Server
To add a new LLM server:
- Go to Settings → LLM Servers.
- Click Add Server.
- Fill in the required fields:
| Field | Description |
|---|---|
| Name | A unique, descriptive name for this connection (e.g. OpenAI Production) |
| Provider | The AI provider type |
| URL | The base URL of the API endpoint |
| Authentication | How to authenticate with the provider (see below) |
- Click Save.
Once saved, you can sync models from this server.
Authentication
The authentication method required depends on the provider:
API Key
Used by cloud providers (OpenAI, Azure OpenAI).
| Field | Description |
|---|---|
| API Key | The secret key issued by the provider |
You can find your API key in your provider’s dashboard.
Basic Auth
Used by self-hosted providers (Ollama).
| Field | Description |
|---|---|
| Username | The username configured on the server |
| Password | The password configured on the server |
If your Ollama instance does not require authentication, you can leave these blank or use placeholder values depending on your setup.
Security note: Credentials are stored securely and are never shown in plain text after saving.
Editing and Deleting Servers
- Edit — You can update the URL, name, or credentials of a server at any time from the server list.
- Delete — Deleting a server permanently removes it and all models associated with it. Any agents using those models will lose their model configuration.
Warning: Deleting an LLM server cannot be undone and will affect all models and agents that depend on it.
LLM Models
Models are the specific AI models available on a connected server. After adding a server, you sync its models to make them available in Evolvable.ai, then enable the ones you want to use.
Model Types
Every model has a type that determines what it can be used for:
| Type | Description |
|---|---|
| Chat | Generates conversational responses. Used to power agents. |
| Embedding | Converts text into vector representations. Used by the knowledge base for document search. |
Syncing Models
After adding a server, you need to sync its models:
- Go to Settings → LLM Models.
- Click Sync Models.
- Evolvable.ai connects to each configured server and fetches the list of available models.
Newly discovered models are added to the list but are disabled by default. You must explicitly enable the models you want to use.
Note: Sync can be re-run at any time to pick up new models added to a server.
Enabling and Disabling Models
Only enabled models can be selected for use in agents and knowledge base settings.
To enable or disable models:
- In the model list, select one or more models.
- Click Enable or Disable.
You can filter the list by:
- Type — Chat or Embedding
- Status — Enabled or Disabled
- Server — Filter by a specific LLM server
- Search — Filter by model name
Embedding Settings
Embedding models have an additional set of settings that control how documents are chunked and stored in the vector database. These settings apply when the model is used in the knowledge base.
| Setting | Description |
|---|---|
| Chunk Size | Target number of tokens per document chunk |
| Min Chunk Size (chars) | Minimum character length — shorter chunks are discarded |
| Min Chunk Length to Embed | Minimum number of characters required before a chunk is embedded |
| Max Chunks | Hard cap on the number of chunks per document |
| Top K | Number of similar chunks to retrieve per query |
| Distance Type | Similarity metric: Cosine, Euclidean, or Manhattan |
These settings can be updated from the model’s detail view. Changing them after documents have already been processed requires reprocessing those documents.
Using Models in Agents
Once a model is enabled, it becomes available when configuring an agent:
- Chat model — set under the agent’s Language Model settings. Controls which model generates the agent’s responses.
- Embedding model — set under the knowledge base Embedding Settings. Controls how documents are vectorized for retrieval.
See Agencies & Agents for more on agent configuration, and Knowledge Base for embedding settings.
Tips and Constraints
- Server names must be unique across Evolvable.ai.
- A server must have at least one enabled model before it can be used by an agent.
- Deleting a server removes all its models — update any agents using those models first.
- Changing an embedding model’s chunk size settings requires reprocessing any documents that used the old settings.
- Models synced from a server cannot be created manually — they are discovered automatically via the sync process.