feat!: breaking change - OpenRouter models are now read from conf/openrouter_models.json while Custom / Self-hosted models are read from conf/custom_models.json
feat: Azure OpenAI / Azure AI Foundry support. Models should be defined in conf/azure_models.json (or a custom path). See .env.example for environment variables or see readme. https://github.com/BeehiveInnovations/zen-mcp-server/issues/265 feat: OpenRouter / Custom Models / Azure can separately also use custom config paths now (see .env.example ) refactor: Model registry class made abstract, OpenRouter / Custom Provider / Azure OpenAI now subclass these refactor: breaking change: `is_custom` property has been removed from model_capabilities.py (and thus custom_models.json) given each models are now read from separate configuration files
This commit is contained in:
@@ -9,6 +9,7 @@ Each provider:
|
||||
- Defines supported models using `ModelCapabilities` objects
|
||||
- Implements the minimal abstract hooks (`get_provider_type()` and `generate_content()`)
|
||||
- Gets wired into `configure_providers()` so environment variables control activation
|
||||
- Can leverage helper subclasses (e.g., `AzureOpenAIProvider`) when only client wiring differs
|
||||
|
||||
### Intelligence score cheatsheet
|
||||
|
||||
@@ -31,6 +32,13 @@ features ([details here](model_ranking.md)).
|
||||
|
||||
⚠️ **Important**: If you implement a custom `generate_content()`, call `_resolve_model_name()` before invoking the SDK so aliases (e.g. `"gpt"` → `"gpt-4"`) resolve correctly. The shared implementations already do this for you.
|
||||
|
||||
**Option C: Azure OpenAI (`AzureOpenAIProvider`)**
|
||||
- For Azure-hosted deployments of OpenAI models
|
||||
- Reuses the OpenAI-compatible pipeline but swaps in the `AzureOpenAI` client and a deployment mapping (canonical model → deployment ID)
|
||||
- Define deployments in [`conf/azure_models.json`](../conf/azure_models.json) (or the file referenced by `AZURE_MODELS_CONFIG_PATH`).
|
||||
- Entries follow the [`ModelCapabilities`](../providers/shared/model_capabilities.py) schema and must include a `deployment` identifier.
|
||||
See [Azure OpenAI Configuration](azure_openai.md) for a step-by-step walkthrough.
|
||||
|
||||
## Step-by-Step Guide
|
||||
|
||||
### 1. Add Provider Type
|
||||
@@ -227,6 +235,19 @@ DISABLED_TOOLS=debug,tracer
|
||||
EXAMPLE_ALLOWED_MODELS=example-model-large,example-model-small
|
||||
```
|
||||
|
||||
For Azure OpenAI deployments:
|
||||
|
||||
```bash
|
||||
AZURE_OPENAI_API_KEY=your_azure_openai_key_here
|
||||
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
|
||||
# Models are defined in conf/azure_models.json (or AZURE_MODELS_CONFIG_PATH)
|
||||
# AZURE_OPENAI_API_VERSION=2024-02-15-preview
|
||||
# AZURE_OPENAI_ALLOWED_MODELS=gpt-4o,gpt-4o-mini
|
||||
# AZURE_MODELS_CONFIG_PATH=/absolute/path/to/custom_azure_models.json
|
||||
```
|
||||
|
||||
You can also define Azure models in [`conf/azure_models.json`](../conf/azure_models.json) (the bundled file is empty so you can copy it safely). Each entry mirrors the `ModelCapabilities` schema and must include a `deployment` field. Set `AZURE_MODELS_CONFIG_PATH` if you maintain a custom copy outside the repository.
|
||||
|
||||
**Note**: The `description` field in `ModelCapabilities` helps Claude choose the best model in auto mode.
|
||||
|
||||
### 5. Test Your Provider
|
||||
|
||||
@@ -91,8 +91,8 @@ OPENAI_ALLOWED_MODELS=o3,o4-mini
|
||||
|
||||
**Important Notes:**
|
||||
- Restrictions apply to all usage including auto mode
|
||||
- `OPENROUTER_ALLOWED_MODELS` only affects OpenRouter models accessed via custom provider (where `is_custom: false` in custom_models.json)
|
||||
- Custom local models (`is_custom: true`) are not affected by any restrictions
|
||||
- `OPENROUTER_ALLOWED_MODELS` only affects models defined in `conf/openrouter_models.json`
|
||||
- Custom local models (from `conf/custom_models.json`) are not affected by OpenRouter restrictions
|
||||
|
||||
## Thinking Modes
|
||||
|
||||
|
||||
62
docs/azure_openai.md
Normal file
62
docs/azure_openai.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# Azure OpenAI Configuration
|
||||
|
||||
Azure OpenAI support lets Zen MCP talk to GPT-4o, GPT-4.1, GPT-5, and o-series deployments that you expose through your Azure resource. This guide describes the configuration expected by the server: a couple of required environment variables plus a JSON manifest that lists every deployment you want to expose.
|
||||
|
||||
## 1. Required Environment Variables
|
||||
|
||||
Set these entries in your `.env` (or MCP `env` block).
|
||||
|
||||
```bash
|
||||
AZURE_OPENAI_API_KEY=your_azure_openai_key_here
|
||||
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
|
||||
# AZURE_OPENAI_API_VERSION=2024-02-15-preview
|
||||
```
|
||||
|
||||
Without the key and endpoint the provider is skipped entirely. Leave the key blank only if the endpoint truly allows anonymous access (rare for Azure).
|
||||
|
||||
## 2. Define Deployments in `conf/azure_models.json`
|
||||
|
||||
Azure models live in `conf/azure_models.json` (or the file pointed to by `AZURE_MODELS_CONFIG_PATH`). Each entry follows the same schema as [`ModelCapabilities`](../providers/shared/model_capabilities.py) with one additional required key: `deployment`. This field must exactly match the deployment name shown in the Azure Portal (for example `prod-gpt4o`). The provider routes requests by that value, so omitting it or using the wrong name will cause the server to skip the model.
|
||||
|
||||
```json
|
||||
{
|
||||
"models": [
|
||||
{
|
||||
"model_name": "gpt-4o",
|
||||
"deployment": "prod-gpt4o",
|
||||
"friendly_name": "Azure GPT-4o EU",
|
||||
"intelligence_score": 18,
|
||||
"context_window": 600000,
|
||||
"max_output_tokens": 128000,
|
||||
"supports_temperature": false,
|
||||
"temperature_constraint": "fixed",
|
||||
"aliases": ["gpt4o-eu"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Tips:
|
||||
|
||||
- Copy `conf/azure_models.json` into your repo and commit it, or point `AZURE_MODELS_CONFIG_PATH` at a custom path.
|
||||
- Add one object per deployment. Aliases are optional but help when you want short names like `gpt4o-eu`.
|
||||
- All capability fields are optional except `model_name`, `deployment`, and `friendly_name`. Anything you omit falls back to conservative defaults.
|
||||
|
||||
## 3. Optional Restrictions
|
||||
|
||||
Use `AZURE_OPENAI_ALLOWED_MODELS` to limit which Azure models Claude can access:
|
||||
|
||||
```bash
|
||||
AZURE_OPENAI_ALLOWED_MODELS=gpt-4o,gpt-4o-mini
|
||||
```
|
||||
|
||||
Aliases are matched case-insensitively.
|
||||
|
||||
## 4. Quick Checklist
|
||||
|
||||
- [ ] `AZURE_OPENAI_API_KEY` and `AZURE_OPENAI_ENDPOINT` are set
|
||||
- [ ] `conf/azure_models.json` (or the file referenced by `AZURE_MODELS_CONFIG_PATH`) lists every deployment with the desired metadata
|
||||
- [ ] Optional: `AZURE_OPENAI_ALLOWED_MODELS` to restrict usage
|
||||
- [ ] Restart `./run-server.sh` and run `listmodels` to confirm the Azure entries appear with the expected metadata
|
||||
|
||||
See also: [`docs/adding_providers.md`](adding_providers.md) for the full provider architecture and [README (Provider Configuration)](../README.md#provider-configuration) for quick-start environment snippets.
|
||||
@@ -158,6 +158,8 @@ XAI_ALLOWED_MODELS=grok,grok-3-fast
|
||||
```env
|
||||
# Override default location of custom_models.json
|
||||
CUSTOM_MODELS_CONFIG_PATH=/path/to/your/custom_models.json
|
||||
# Override default location of openrouter_models.json
|
||||
OPENROUTER_MODELS_CONFIG_PATH=/path/to/your/openrouter_models.json
|
||||
```
|
||||
|
||||
**Conversation Settings:**
|
||||
@@ -244,4 +246,4 @@ LOG_LEVEL=INFO
|
||||
|
||||
- **[Advanced Usage Guide](advanced-usage.md)** - Advanced model usage patterns, thinking modes, and power user workflows
|
||||
- **[Context Revival Guide](context-revival.md)** - Conversation persistence and context revival across sessions
|
||||
- **[AI-to-AI Collaboration Guide](ai-collaboration.md)** - Multi-model coordination and conversation threading
|
||||
- **[AI-to-AI Collaboration Guide](ai-collaboration.md)** - Multi-model coordination and conversation threading
|
||||
|
||||
@@ -35,7 +35,12 @@ This guide covers setting up multiple AI model providers including OpenRouter, c
|
||||
|
||||
## Model Aliases
|
||||
|
||||
The server uses `conf/custom_models.json` to map convenient aliases to both OpenRouter and custom model names. This unified registry supports both cloud models (via OpenRouter) and local models (via custom endpoints).
|
||||
Zen ships two registries:
|
||||
|
||||
- `conf/openrouter_models.json` – metadata for models routed through OpenRouter. Override with `OPENROUTER_MODELS_CONFIG_PATH` if you maintain a custom copy.
|
||||
- `conf/custom_models.json` – metadata for local or self-hosted OpenAI-compatible endpoints used by the Custom provider. Override with `CUSTOM_MODELS_CONFIG_PATH` if needed.
|
||||
|
||||
Copy whichever file you need into your project (or point the corresponding `*_MODELS_CONFIG_PATH` env var at your own copy) and edit it to advertise the models you want.
|
||||
|
||||
### OpenRouter Models (Cloud)
|
||||
|
||||
@@ -58,7 +63,7 @@ The server uses `conf/custom_models.json` to map convenient aliases to both Open
|
||||
|-------|-------------------|------|
|
||||
| `local-llama`, `local` | `llama3.2` | Requires `CUSTOM_API_URL` configured |
|
||||
|
||||
View the full list in [`conf/custom_models.json`](conf/custom_models.json).
|
||||
View the baseline OpenRouter catalogue in [`conf/openrouter_models.json`](conf/openrouter_models.json) and populate [`conf/custom_models.json`](conf/custom_models.json) with your local models.
|
||||
|
||||
To control ordering in auto mode or the `listmodels` summary, adjust the
|
||||
[`intelligence_score`](model_ranking.md) for each entry (or rely on the automatic
|
||||
@@ -152,7 +157,7 @@ CUSTOM_MODEL_NAME=your-loaded-model
|
||||
|
||||
## Using Models
|
||||
|
||||
**Using model aliases (from conf/custom_models.json):**
|
||||
**Using model aliases (from the registry files):**
|
||||
```
|
||||
# OpenRouter models:
|
||||
"Use opus for deep analysis" # → anthropic/claude-opus-4
|
||||
@@ -185,20 +190,20 @@ CUSTOM_MODEL_NAME=your-loaded-model
|
||||
|
||||
The system automatically routes models to the appropriate provider:
|
||||
|
||||
1. **Models with `is_custom: true`** → Always routed to Custom API (requires `CUSTOM_API_URL`)
|
||||
2. **Models with `is_custom: false` or omitted** → Routed to OpenRouter (requires `OPENROUTER_API_KEY`)
|
||||
1. Entries in `conf/custom_models.json` → Always routed through the Custom API (requires `CUSTOM_API_URL`)
|
||||
2. Entries in `conf/openrouter_models.json` → Routed through OpenRouter (requires `OPENROUTER_API_KEY`)
|
||||
3. **Unknown models** → Fallback logic based on model name patterns
|
||||
|
||||
**Provider Priority Order:**
|
||||
1. Native APIs (Google, OpenAI) - if API keys are available
|
||||
2. Custom endpoints - for models marked with `is_custom: true`
|
||||
2. Custom endpoints - for models declared in `conf/custom_models.json`
|
||||
3. OpenRouter - catch-all for cloud models
|
||||
|
||||
This ensures clean separation between local and cloud models while maintaining flexibility for unknown models.
|
||||
|
||||
## Model Configuration
|
||||
|
||||
The server uses `conf/custom_models.json` to define model aliases and capabilities. You can:
|
||||
These JSON files define model aliases and capabilities. You can:
|
||||
|
||||
1. **Use the default configuration** - Includes popular models with convenient aliases
|
||||
2. **Customize the configuration** - Add your own models and aliases
|
||||
@@ -206,7 +211,7 @@ The server uses `conf/custom_models.json` to define model aliases and capabiliti
|
||||
|
||||
### Adding Custom Models
|
||||
|
||||
Edit `conf/custom_models.json` to add new models. The configuration supports both OpenRouter (cloud) and custom endpoint (local) models.
|
||||
Edit `conf/openrouter_models.json` to tweak OpenRouter behaviour or `conf/custom_models.json` to add local models. Each entry maps directly onto [`ModelCapabilities`](../providers/shared/model_capabilities.py).
|
||||
|
||||
#### Adding an OpenRouter Model
|
||||
|
||||
@@ -232,7 +237,6 @@ Edit `conf/custom_models.json` to add new models. The configuration supports bot
|
||||
"supports_extended_thinking": false,
|
||||
"supports_json_mode": false,
|
||||
"supports_function_calling": false,
|
||||
"is_custom": true,
|
||||
"description": "My custom Ollama/vLLM model"
|
||||
}
|
||||
```
|
||||
@@ -244,10 +248,9 @@ Edit `conf/custom_models.json` to add new models. The configuration supports bot
|
||||
- `supports_extended_thinking`: Whether the model has extended reasoning capabilities
|
||||
- `supports_json_mode`: Whether the model can guarantee valid JSON output
|
||||
- `supports_function_calling`: Whether the model supports function/tool calling
|
||||
- `is_custom`: **Set to `true` for models that should ONLY work with custom endpoints** (Ollama, vLLM, etc.)
|
||||
- `description`: Human-readable description of the model
|
||||
|
||||
**Important:** Always set `is_custom: true` for local models. This ensures they're only used when `CUSTOM_API_URL` is configured and prevents conflicts with OpenRouter.
|
||||
**Important:** Keep OpenRouter and Custom models in their respective files so that requests are routed correctly.
|
||||
|
||||
## Available Models
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
|----------|-------------|
|
||||
| [Getting Started](getting-started.md) | Installation paths, prerequisite setup, and first-run guidance. |
|
||||
| [Adding Providers](adding_providers.md) | How to register new AI providers and advertise capabilities. |
|
||||
| [Azure OpenAI](azure_openai.md) | Configure Azure deployments, capability overrides, and env mappings. |
|
||||
| [Model Ranking](model_ranking.md) | How intelligence scores translate into auto-mode ordering. |
|
||||
| [Custom Models](custom_models.md) | Configure OpenRouter/custom models and aliases. |
|
||||
| [Adding Tools](adding_tools.md) | Create new tools using the shared base classes. |
|
||||
|
||||
@@ -25,7 +25,7 @@ feature_bonus = (
|
||||
+ (1 if supports_json_mode else 0)
|
||||
+ (1 if supports_images else 0)
|
||||
)
|
||||
penalty = 1 if is_custom else 0
|
||||
penalty = 1 if provider == CUSTOM else 0
|
||||
|
||||
effective_rank = clamp(base + ctx_bonus + output_bonus + feature_bonus - penalty, 0, 100)
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user