torbjorn/my-pal-mcp-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WIP

Browse Source 
- OpenRouter model configuration registry
- Model definition file for users to be able to control
- Update instructions
This commit is contained in:
Fahad
2025-06-13 05:52:16 +04:00
parent 93daa2942a
commit a19055b76a
7 changed files with 291 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -107,8 +107,8 @@ The final implementation resulted in a 26% improvement in JSON parsing performan
**Option B: OpenRouter (Access multiple models with one API)**
- **OpenRouter**: Visit [OpenRouter](https://openrouter.ai/) for access to multiple models through one API. [Setup Guide](docs/openrouter.md)
- Set `OPENROUTER_ALLOWED_MODELS` to restrict which models can be used (recommended)
- Leave empty to allow ANY model (warning: some models are expensive!)
- Control model access and spending limits directly in your OpenRouter dashboard
- Configure model aliases in `conf/openrouter_models.json`
> **Note:** Using both OpenRouter and native APIs creates ambiguity about which provider serves each model. If both are configured, native APIs will take priority.

6
config.py
View File

@@ -59,6 +59,12 @@ MODEL_CAPABILITIES_DESC = {
"gemini-2.5-pro-preview-06-05": "Deep reasoning + thinking mode (1M context) - Complex problems, architecture, deep analysis",
}
# Note: When only OpenRouter is configured, these model aliases automatically map to equivalent models:
# - "flash" → "google/gemini-flash-1.5-8b"
# - "pro" → "google/gemini-pro-1.5"
# - "o3" → "openai/gpt-4o"
# - "o3-mini" → "openai/gpt-4o-mini"
# Token allocation for Gemini Pro (1M total capacity)
# MAX_CONTEXT_TOKENS: Total model capacity
# MAX_CONTENT_TOKENS: Available for prompts, conversation history, and files

2
docker-compose.yml
View File

@@ -33,7 +33,7 @@ services:
- OPENAI_API_KEY=${OPENAI_API_KEY:-}
# OpenRouter support
- OPENROUTER_API_KEY=${OPENROUTER_API_KEY:-}
- OPENROUTER_ALLOWED_MODELS=${OPENROUTER_ALLOWED_MODELS:-}
- OPENROUTER_MODELS_PATH=${OPENROUTER_MODELS_PATH:-}
- DEFAULT_MODEL=${DEFAULT_MODEL:-auto}
- DEFAULT_THINKING_MODE_THINKDEEP=${DEFAULT_THINKING_MODE_THINKDEEP:-high}
- REDIS_URL=redis://redis:6379/0

83
docs/openrouter.md
View File

@@ -16,6 +16,27 @@ OpenRouter provides unified access to multiple AI models (GPT-4, Claude, Mistral
**Important:** Don't use both OpenRouter and native APIs simultaneously - this creates ambiguity about which provider serves each model.
## Model Aliases
The server uses `conf/openrouter_models.json` to map convenient aliases to OpenRouter model names. Some popular aliases:
| Alias | Maps to OpenRouter Model |
|-------|-------------------------|
| `opus` | `anthropic/claude-3-opus` |
| `sonnet`, `claude` | `anthropic/claude-3-sonnet` |
| `haiku` | `anthropic/claude-3-haiku` |
| `gpt4o`, `4o` | `openai/gpt-4o` |
| `gpt4o-mini`, `4o-mini` | `openai/gpt-4o-mini` |
| `gemini`, `pro-openrouter` | `google/gemini-pro-1.5` |
| `flash-openrouter` | `google/gemini-flash-1.5-8b` |
| `mistral` | `mistral/mistral-large` |
| `deepseek`, `coder` | `deepseek/deepseek-coder` |
| `perplexity` | `perplexity/llama-3-sonar-large-32k-online` |
View the full list in `conf/openrouter_models.json`.
**Note:** While you can use any OpenRouter model by its full name, models not in the config file will use generic capabilities (32K context window, no extended thinking, etc.) which may not match the model's actual capabilities. For best results, add new models to the config file with their proper specifications.
## Quick Start
### 1. Get API Key
@@ -27,37 +48,63 @@ OpenRouter provides unified access to multiple AI models (GPT-4, Claude, Mistral
```bash
# Add to your .env file
OPENROUTER_API_KEY=your-openrouter-api-key
# IMPORTANT: Set allowed models to control costs
OPENROUTER_ALLOWED_MODELS=gpt-4,claude-3-sonnet,mistral-large
# Or leave empty to allow ANY model (WARNING: risk of high costs!)
# OPENROUTER_ALLOWED_MODELS=
```
> **Note:** Control which models can be used directly in your OpenRouter dashboard at [openrouter.ai](https://openrouter.ai/). This gives you centralized control over model access and spending limits.
That's it! Docker Compose already includes all necessary configuration.
### 3. Use Models
**If you set OPENROUTER_ALLOWED_MODELS:**
**Using model aliases (from conf/openrouter_models.json):**
```
# Only these models will work:
"Use gpt-4 via zen to review this code"
"Use claude-3-sonnet via zen to debug this error"
"Use mistral-large via zen to optimize this algorithm"
# Use short aliases:
"Use opus via zen for deep analysis" # → anthropic/claude-3-opus
"Use sonnet via zen to review this code" # → anthropic/claude-3-sonnet
"Use gpt4o via zen to analyze this" # → openai/gpt-4o
"Use mistral via zen to optimize" # → mistral/mistral-large
```
**If you leave OPENROUTER_ALLOWED_MODELS empty:**
**Using full model names:**
```
# ANY model available on OpenRouter will work:
"Use gpt-4o via zen to analyze this"
"Use claude-3-opus via zen for deep analysis"
"Use deepseek-coder via zen to generate code"
# WARNING: Some models can be very expensive!
# Any model available on OpenRouter:
"Use anthropic/claude-3-opus via zen for deep analysis"
"Use openai/gpt-4o via zen to debug this"
"Use deepseek/deepseek-coder via zen to generate code"
```
Check current model pricing at [openrouter.ai/models](https://openrouter.ai/models).
## Model Configuration
The server uses `conf/openrouter_models.json` to define model aliases and capabilities. You can:
1. **Use the default configuration** - Includes popular models with convenient aliases
2. **Customize the configuration** - Add your own models and aliases
3. **Override the config path** - Set `OPENROUTER_MODELS_PATH` environment variable
### Adding Custom Models
Edit `conf/openrouter_models.json` to add new models:
```json
{
"model_name": "vendor/model-name",
"aliases": ["short-name", "nickname"],
"context_window": 128000,
"supports_extended_thinking": false,
"supports_json_mode": true,
"supports_function_calling": true,
"description": "Model description"
}
```
**Field explanations:**
- `context_window`: Total tokens the model can process (input + output combined)
- `supports_extended_thinking`: Whether the model has extended reasoning capabilities
- `supports_json_mode`: Whether the model can guarantee valid JSON output
- `supports_function_calling`: Whether the model supports function/tool calling
## Available Models
Popular models available through OpenRouter:
@@ -71,4 +118,4 @@ Popular models available through OpenRouter:
- **"Model not found"**: Check exact model name at openrouter.ai/models
- **"Insufficient credits"**: Add credits to your OpenRouter account
- **"Model not in allow-list"**: Update `OPENROUTER_ALLOWED_MODELS` in .env
- **"Model not available"**: Check your OpenRouter dashboard for model access permissions

160
providers/openrouter.py
View File

@@ -2,13 +2,16 @@
import logging
import os
from typing import Optional
from .base import (
ModelCapabilities,
ModelResponse,
ProviderType,
RangeTemperatureConstraint,
)
from .openai_compatible import OpenAICompatibleProvider
from .openrouter_registry import OpenRouterModelRegistry
class OpenRouterProvider(OpenAICompatibleProvider):
@@ -26,6 +29,9 @@ class OpenRouterProvider(OpenAICompatibleProvider):
"X-Title": os.getenv("OPENROUTER_TITLE", "Zen MCP Server"),
}
# Model registry for managing configurations and aliases
_registry: Optional[OpenRouterModelRegistry] = None
def __init__(self, api_key: str, **kwargs):
"""Initialize OpenRouter provider.
@@ -36,49 +42,87 @@ class OpenRouterProvider(OpenAICompatibleProvider):
# Always use OpenRouter's base URL
super().__init__(api_key, base_url="https://openrouter.ai/api/v1", **kwargs)
# Log warning about model allow-list if not configured
if not self.allowed_models:
logging.warning(
"OpenRouter provider initialized without model allow-list. "
"Consider setting OPENROUTER_ALLOWED_MODELS environment variable "
"to restrict model access and control costs."
)
# Initialize model registry
if OpenRouterProvider._registry is None:
OpenRouterProvider._registry = OpenRouterModelRegistry()
# Log loaded models and aliases
models = self._registry.list_models()
aliases = self._registry.list_aliases()
logging.info(
f"OpenRouter loaded {len(models)} models with {len(aliases)} aliases"
)
def _parse_allowed_models(self) -> None:
"""Override to disable environment-based allow-list.
OpenRouter model access is controlled via the OpenRouter dashboard,
not through environment variables.
"""
return None
def _resolve_model_name(self, model_name: str) -> str:
"""Resolve model aliases to OpenRouter model names.
Args:
model_name: Input model name or alias
Returns:
Resolved OpenRouter model name
"""
# Try to resolve through registry
config = self._registry.resolve(model_name)
if config:
if config.model_name != model_name:
logging.info(f"Resolved model alias '{model_name}' to '{config.model_name}'")
return config.model_name
else:
# If not found in registry, return as-is
# This allows using models not in our config file
logging.debug(f"Model '{model_name}' not found in registry, using as-is")
return model_name
def get_capabilities(self, model_name: str) -> ModelCapabilities:
"""Get capabilities for a model.
Since OpenRouter supports many models dynamically, we return
generic capabilities with conservative defaults.
Args:
model_name: Name of the model
model_name: Name of the model (or alias)
Returns:
Generic ModelCapabilities with warnings logged
ModelCapabilities from registry or generic defaults
"""
logging.warning(
f"Using generic capabilities for '{model_name}' via OpenRouter. "
"Actual model capabilities may differ. Consider querying OpenRouter's "
"/models endpoint for accurate information."
)
# Try to get from registry first
capabilities = self._registry.get_capabilities(model_name)
# Create generic capabilities with conservative defaults
capabilities = ModelCapabilities(
provider=ProviderType.OPENROUTER,
model_name=model_name,
friendly_name=self.FRIENDLY_NAME,
max_tokens=32_768, # Conservative default
supports_extended_thinking=False, # Most models don't support this
supports_system_prompts=True, # Most models support this
supports_streaming=True,
supports_function_calling=False, # Varies by model
temperature_constraint=RangeTemperatureConstraint(0.0, 2.0, 1.0),
)
# Mark as generic for validation purposes
capabilities._is_generic = True
return capabilities
if capabilities:
return capabilities
else:
# Resolve any potential aliases and create generic capabilities
resolved_name = self._resolve_model_name(model_name)
logging.debug(
f"Using generic capabilities for '{resolved_name}' via OpenRouter. "
"Consider adding to openrouter_models.json for specific capabilities."
)
# Create generic capabilities with conservative defaults
capabilities = ModelCapabilities(
provider=ProviderType.OPENROUTER,
model_name=resolved_name,
friendly_name=self.FRIENDLY_NAME,
max_tokens=32_768, # Conservative default context window
supports_extended_thinking=False,
supports_system_prompts=True,
supports_streaming=True,
supports_function_calling=False,
temperature_constraint=RangeTemperatureConstraint(0.0, 2.0, 1.0),
)
# Mark as generic for validation purposes
capabilities._is_generic = True
return capabilities
def get_provider_type(self) -> ProviderType:
"""Get the provider type."""
@@ -87,23 +131,53 @@ class OpenRouterProvider(OpenAICompatibleProvider):
def validate_model_name(self, model_name: str) -> bool:
"""Validate if the model name is allowed.
For OpenRouter, we accept any model name unless an allow-list
is configured via OPENROUTER_ALLOWED_MODELS environment variable.
For OpenRouter, we accept any model name. OpenRouter will
validate based on the API key's permissions.
Args:
model_name: Model name to validate
Returns:
True if model is allowed, False otherwise
Always True - OpenRouter handles validation
"""
if self.allowed_models:
# Case-insensitive validation against allow-list
return model_name.lower() in self.allowed_models
# Accept any model if no allow-list configured
# The API will return an error if the model doesn't exist
# Accept any model name - OpenRouter will validate based on API key permissions
return True
def generate_content(
self,
prompt: str,
model_name: str,
system_prompt: Optional[str] = None,
temperature: float = 0.7,
max_output_tokens: Optional[int] = None,
**kwargs,
) -> ModelResponse:
"""Generate content using the OpenRouter API.
Args:
prompt: User prompt to send to the model
model_name: Name of the model (or alias) to use
system_prompt: Optional system prompt for model behavior
temperature: Sampling temperature
max_output_tokens: Maximum tokens to generate
**kwargs: Additional provider-specific parameters
Returns:
ModelResponse with generated content and metadata
"""
# Resolve model alias to actual OpenRouter model name
resolved_model = self._resolve_model_name(model_name)
# Call parent method with resolved model name
return super().generate_content(
prompt=prompt,
model_name=resolved_model,
system_prompt=system_prompt,
temperature=temperature,
max_output_tokens=max_output_tokens,
**kwargs
)
def supports_thinking_mode(self, model_name: str) -> bool:
"""Check if the model supports extended thinking mode.

6
server.py
View File

@@ -549,11 +549,7 @@ async def handle_get_version() -> list[TextContent]:
if ModelProviderRegistry.get_provider(ProviderType.OPENAI):
configured_providers.append("OpenAI (o3, o3-mini)")
if ModelProviderRegistry.get_provider(ProviderType.OPENROUTER):
openrouter_allowed = os.getenv("OPENROUTER_ALLOWED_MODELS", "")
if openrouter_allowed:
configured_providers.append(f"OpenRouter (restricted to: {openrouter_allowed})")
else:
configured_providers.append("OpenRouter (ANY model on openrouter.ai)")
configured_providers.append("OpenRouter (configured via conf/openrouter_models.json)")
# Format the information in a human-readable way
text = f"""Zen MCP Server v{__version__}

123
tests/test_openrouter_provider.py
View File

@@ -38,44 +38,62 @@ class TestOpenRouterProvider:
assert provider.DEFAULT_HEADERS["HTTP-Referer"] == "https://myapp.com"
assert provider.DEFAULT_HEADERS["X-Title"] == "My App"
def test_model_validation_without_allowlist(self):
"""Test model validation without allow-list."""
def test_model_validation(self):
"""Test model validation."""
provider = OpenRouterProvider(api_key="test-key")
# Should accept any model when no allow-list
# Should accept any model - OpenRouter handles validation
assert provider.validate_model_name("gpt-4") is True
assert provider.validate_model_name("claude-3-opus") is True
assert provider.validate_model_name("any-model-name") is True
def test_model_validation_with_allowlist(self):
"""Test model validation with allow-list."""
with patch.dict(os.environ, {
"OPENROUTER_ALLOWED_MODELS": "gpt-4,claude-3-opus,mistral-large"
}):
provider = OpenRouterProvider(api_key="test-key")
# Test allowed models (case-insensitive)
assert provider.validate_model_name("gpt-4") is True
assert provider.validate_model_name("GPT-4") is True
assert provider.validate_model_name("claude-3-opus") is True
assert provider.validate_model_name("MISTRAL-LARGE") is True
# Test disallowed models
assert provider.validate_model_name("gpt-3.5-turbo") is False
assert provider.validate_model_name("unauthorized-model") is False
assert provider.validate_model_name("GPT-4") is True
assert provider.validate_model_name("unknown-model") is True
def test_get_capabilities(self):
"""Test capability generation returns generic capabilities."""
"""Test capability generation."""
provider = OpenRouterProvider(api_key="test-key")
# Should return generic capabilities for any model
caps = provider.get_capabilities("gpt-4")
# Test with a model in the registry (using alias)
caps = provider.get_capabilities("gpt4o")
assert caps.provider == ProviderType.OPENROUTER
assert caps.model_name == "gpt-4"
assert caps.model_name == "openai/gpt-4o" # Resolved name
assert caps.friendly_name == "OpenRouter"
# Test with a model not in registry - should get generic capabilities
caps = provider.get_capabilities("unknown-model")
assert caps.provider == ProviderType.OPENROUTER
assert caps.model_name == "unknown-model"
assert caps.max_tokens == 32_768 # Safe default
assert hasattr(caps, '_is_generic') and caps._is_generic is True
def test_model_alias_resolution(self):
"""Test model alias resolution."""
provider = OpenRouterProvider(api_key="test-key")
# Test alias resolution
assert provider._resolve_model_name("opus") == "anthropic/claude-3-opus"
assert provider._resolve_model_name("sonnet") == "anthropic/claude-3-sonnet"
assert provider._resolve_model_name("gpt4o") == "openai/gpt-4o"
assert provider._resolve_model_name("4o") == "openai/gpt-4o"
assert provider._resolve_model_name("claude") == "anthropic/claude-3-sonnet"
assert provider._resolve_model_name("mistral") == "mistral/mistral-large"
assert provider._resolve_model_name("deepseek") == "deepseek/deepseek-coder"
assert provider._resolve_model_name("coder") == "deepseek/deepseek-coder"
# Test case-insensitive
assert provider._resolve_model_name("OPUS") == "anthropic/claude-3-opus"
assert provider._resolve_model_name("GPT4O") == "openai/gpt-4o"
assert provider._resolve_model_name("Mistral") == "mistral/mistral-large"
assert provider._resolve_model_name("CLAUDE") == "anthropic/claude-3-sonnet"
# Test direct model names (should pass through unchanged)
assert provider._resolve_model_name("anthropic/claude-3-opus") == "anthropic/claude-3-opus"
assert provider._resolve_model_name("openai/gpt-4o") == "openai/gpt-4o"
# Test unknown models pass through
assert provider._resolve_model_name("unknown-model") == "unknown-model"
assert provider._resolve_model_name("custom/model-v2") == "custom/model-v2"
def test_openrouter_registration(self):
"""Test OpenRouter can be registered and retrieved."""
with patch.dict(os.environ, {"OPENROUTER_API_KEY": "test-key"}):
@@ -91,6 +109,63 @@ class TestOpenRouterProvider:
assert isinstance(provider, OpenRouterProvider)
class TestOpenRouterRegistry:
"""Test cases for OpenRouter model registry."""
def test_registry_loading(self):
"""Test registry loads models from config."""
from providers.openrouter_registry import OpenRouterModelRegistry
registry = OpenRouterModelRegistry()
# Should have loaded models
models = registry.list_models()
assert len(models) > 0
assert "anthropic/claude-3-opus" in models
assert "openai/gpt-4o" in models
# Should have loaded aliases
aliases = registry.list_aliases()
assert len(aliases) > 0
assert "opus" in aliases
assert "gpt4o" in aliases
assert "claude" in aliases
def test_registry_capabilities(self):
"""Test registry provides correct capabilities."""
from providers.openrouter_registry import OpenRouterModelRegistry
registry = OpenRouterModelRegistry()
# Test known model
caps = registry.get_capabilities("opus")
assert caps is not None
assert caps.model_name == "anthropic/claude-3-opus"
assert caps.max_tokens == 200000 # Claude's context window
# Test using full model name
caps = registry.get_capabilities("anthropic/claude-3-opus")
assert caps is not None
assert caps.model_name == "anthropic/claude-3-opus"
# Test unknown model
caps = registry.get_capabilities("non-existent-model")
assert caps is None
def test_multiple_aliases_same_model(self):
"""Test multiple aliases pointing to same model."""
from providers.openrouter_registry import OpenRouterModelRegistry
registry = OpenRouterModelRegistry()
# All these should resolve to Claude Sonnet
sonnet_aliases = ["sonnet", "claude", "claude-sonnet", "claude3-sonnet"]
for alias in sonnet_aliases:
config = registry.resolve(alias)
assert config is not None
assert config.model_name == "anthropic/claude-3-sonnet"
class TestOpenRouterSSRFProtection:
"""Test SSRF protection for OpenRouter."""