Adds flexible cassette matching that ignores system prompt changes
for o3 models, preventing CI failures when prompts are updated.
Changes:
- Semantic matching: Only compares model name, user question, and core params
- Ignores: System prompts, conversation memory instructions, metadata
- Prevents cassette breaks when prompts change between code versions
- Added comprehensive tests for semantic matching behavior
- Created maintenance documentation (tests/CASSETTE_MAINTENANCE.md)
This solves the CI failure where o3-pro test cassettes would break
whenever system prompts or conversation memory format changed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Updated test_supported_models_aliases.py to reflect the removal of self-referencing aliases:
- Removed assertion for "o4-mini" in its own aliases (no longer self-referencing)
- Updated "o3-pro" alias test to use "o3pro" (normalized alias format)
- Fixed alias resolution test for o3pro -> o3-pro
These changes align with the fix for duplicate model listings in listmodels output.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Extract restriction checking logic into reusable helper method
- Refactor validate_model_name to reduce code duplication
- Fix logging import by using existing module-level logger
- Clean up test file by removing print statement and main block
- All tests continue to pass after refactoring
- OpenAI provider now checks custom models registry for user configurations
- Custom models with supports_temperature=false no longer send temperature to API
- Fixes 400 errors for custom o3/gpt-5 models configured without temperature support
- Added comprehensive tests to verify the fix works correctly
- Maintains backward compatibility with built-in models
Fixes#245
Fixed runtime bug where _prepare_file_content_for_prompt was called
without required model_context parameter, causing RuntimeError when
processing requests with relevant_files.
- Create ModelContext instance with model_name in _consult_model method
- Pass model_context parameter to _prepare_file_content_for_prompt call
- Add comprehensive regression test to prevent future occurrences
- Maintain consensus tool's blinded design with independent model contexts
Disabled secondary tools by default (for new installations), updated README.md with instructions on how to enable these in .env
run-server.sh now displays disabled / enabled tools (when DISABLED_TOOLS is set)
fix: Minor tweaks to prompts
fix: Improved support for smaller models that struggle with strict structured JSON output
Rearranged reasons to use the MCP above quick start (collapsed)
This commit updates all references to Claude Opus 4 and Sonnet 4 to their newer 4.1 versions throughout the codebase.
The changes include:
- Updating model names in `conf/custom_models.json` and `providers/dial.py`.
- Updating aliases and descriptions to match the new model versions.
- Updating `.env.example` to reflect the new model names.
- Updating all relevant test suites to use the new model names and ensure all tests pass.
- Fix test_resource_loading_success by removing outdated mock targeting non-existent 'files' import
- Simplify resource loading test to validate registry functionality directly
- Add .zen_venv exclusion to ruff and black in code_quality_checks.sh
- All tests now passing (793/793) with clean linting
- Remove redundant path checks between Path("conf/custom_models.json") and Path.cwd() variants
- Implement proper importlib.resources.files('conf') approach for robust packaging
- Create conf/__init__.py to make conf a proper Python package
- Update pyproject.toml to include conf* in package discovery
- Clean up verbose comments and simplify resource loading logic
- Fix test mocking to use correct importlib.resources.files target
- All tests passing (8/8) with proper resource and fallback functionality
Addresses all gemini-code-assist bot feedback from PR #227
Improvements based on gemini-code-assist bot feedback:
1. **Proper importlib.resources implementation:**
- Use files("providers") / "../conf/custom_models.json" for resource loading
- Prioritize resource loading over file system paths for packaged environments
- Maintain backward compatibility with explicit config paths and env variables
2. **Remove redundant path checks:**
- Eliminated duplicate Path("conf/custom_models.json") and Path.cwd() / "conf/custom_models.json"
- Streamlined fallback logic to development path + working directory only
3. **Enhanced test coverage:**
- Mock-based testing of actual fallback scenarios with Path.exists
- Proper resource loading simulation and failure testing
- Comprehensive coverage of both resource and file system modes
4. **Robust error handling:**
- Graceful fallback from resources to file system when resource loading fails
- Clear logging of which loading method is being used
- Better error messages indicating resource vs file system loading
The implementation now follows Python packaging best practices using importlib.resources
while maintaining full backward compatibility and robust fallback behavior.
Tested: All 8 test cases pass, resource loading works in development,
file system fallback works when resources fail.
Resolves issues #203, #186, #206, #185 where OpenRouter model registry
completely failed to load in uvx installations due to inaccessible
conf/custom_models.json file.
Changes:
- Implement multiple path resolution strategy in OpenRouterModelRegistry
- Development: Path(__file__).parent.parent / "conf" / "custom_models.json"
- UVX working dir: Path("conf/custom_models.json")
- Current working dir: Path.cwd() / "conf" / "custom_models.json"
- Add importlib-resources fallback for Python < 3.9 compatibility
- Add comprehensive test suite for path resolution scenarios
- Ensure graceful handling when config files are missing
The fix restores full OpenRouter functionality (15 models, 62+ aliases)
for users installing via uvx while maintaining backward compatibility
for development and explicit config scenarios.
Tested: All path resolution scenarios pass, OpenRouter models load correctly
- Remove broken test with unused mock parameter
- Replace placeholder test with actual validation of diagnostic messages
- Remove unused imports (MagicMock, patch)
- Fix whitespace and formatting issues
- Ensure all 6 tests pass with meaningful assertions
Addresses high-priority feedback from PR review comments.
Remove 7 empty test methods that contained only 'pass' statements:
- TestPipDetectionPlatformCompatibility (4 methods)
- TestPipDetectionRegression (3 methods)
Keep working tests that have actual logic and assertions.
- Convert virtual environment Python paths to absolute paths to ensure
consistency across different shell environments (Git Bash, WSL, etc.)
- Add enhanced diagnostic information when pip detection fails to help
users troubleshoot path and environment issues
- Improve error messages with specific guidance for different platforms
- Fix black configuration to exclude .zen_venv directory from formatting
- Add comprehensive test suite for pip detection edge cases
Fixes#188
- Use o3-pro throughout the codebase instead of o3-pro-2025-06-10
- Update test expectations to match o3-pro model name
- Update cassette to use o3-pro for consistency
- Ensure responses endpoint routing works correctly with o3-pro
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Use new output_text field format for o3-pro responses
- Update test expectations to use resolved model name o3-pro-2025-06-10
- Keep HTTP transport recorder and PII sanitization improvements
- Preserve both bug fix and recent GPT-5 updates
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Improvements to model name resolution
Improved instructions for multi-step workflows when continuation is available
Improved instructions for chat tool
Improved preferred model resolution, moved code from registry -> each provider
Updated tests
The o3-pro test now clears the restriction service singleton in its
setup_method to ensure it re-reads environment variables set by the
@patch.dict decorator. This prevents cached restrictions from previous
tests (like test_fallback_with_shorthand_restrictions) from blocking
the o3-pro model.
This is a minimal, targeted fix that only affects the specific test
that needs it, without breaking other tests that may depend on the
restriction service state.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace @patch.dict decorator with pytest monkeypatch fixture in
test_fallback_with_shorthand_restrictions to ensure proper environment
variable cleanup between tests. This prevents OPENAI_ALLOWED_MODELS
from leaking into subsequent tests.
Also remove the manual clearing of _restriction_service singleton
as it's no longer needed with proper environment variable isolation.
This fixes test isolation issues where o3-pro tests would fail when
run after restriction tests due to environment variable persistence.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The test failures in CI were caused by tests with @pytest.mark.no_mock_provider
that prevented dummy API keys from being set. In CI with no real API keys,
this led to 'Model not available' errors.
Changed pytest_collection_modifyitems to always set dummy keys if missing,
regardless of markers. This ensures tests work in CI while still allowing
real API keys to be used when present.
Fixes test_conversation_field_mapping.py failures in CI across Python 3.10-3.12.
- Removed excessive debug logging in http_transport_recorder.py
- Consolidated redundant log statements
- Fixed exception logging to use logger.exception()
- Removed emojis from log messages for cleaner output
- Removed __main__ block from test_o3_pro_output_text_fix.py per pytest conventions
- Applied black formatting to comply with CI checks
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix test isolation by clearing LOCALE env var in o3-pro test
- Add restriction service cleanup in test_model_restrictions.py
- Fix PII sanitizer phone regex to not match timestamps
- Convert all print statements to logging in test files per PR review
- Re-record o3-pro cassette with correct environment
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed test_fallback_with_shorthand_restrictions to clear restriction
service singleton in finally block, preventing state leakage
- Updated o3-pro test to use @patch.dict for OPENAI_ALLOWED_MODELS,
following standard pattern and allowing both o3-pro and o3-pro-2025-06-10
- Removed invalid cassette file that had wrong request content
The test now passes in both isolated and full suite runs.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The test was failing when run in the full test suite because the
ModelRestrictionService singleton persisted restrictions from previous
tests. Specifically, test_fallback_with_shorthand_restrictions sets
OPENAI_ALLOWED_MODELS="mini" which blocked o3-pro.
Added utils.model_restrictions._restriction_service = None to ensure
the test starts with clean restriction state.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
The bisect and simplified test files were created during investigation
to understand fixture requirements, but they test the same core functionality
as test_o3_pro_output_text_fix.py. Now that we have the final clean
implementation, these files are redundant.
Removed:
• test_o3_pro_fixture_bisect.py - 4 test methods testing fixture combinations
• test_o3_pro_simplified.py - 2 test methods testing minimal requirements
The main test_o3_pro_output_text_fix.py remains and covers all the necessary
o3-pro output_text parsing validation.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
✨ Key improvements:
• Added public reset_for_testing() method to registry for clean test state management
• Updated test setup/teardown to use new public API instead of private attributes
• Enhanced inject_transport helper to ensure OpenAI provider registration
• Migrated additional test files to use inject_transport pattern
• Reduced code duplication by ~30 lines across test files
🔧 Technical details:
• transport_helpers.py: Always register OpenAI provider for transport tests
• test_o3_pro_output_text_fix.py: Use reset_for_testing() API, remove redundant registration
• test_o3_pro_fixture_bisect.py: Migrate all 4 test methods to inject_transport
• test_o3_pro_simplified.py: Migrate both test methods to inject_transport
• providers/registry.py: Add reset_for_testing() public method
✅ Quality assurance:
• All 7 o3-pro tests pass with new helper pattern
• No regression in test isolation or provider state management
• Improved maintainability through centralized transport injection
• Follows single responsibility principle with focused helper function
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Remove over-engineered allow_all_models fixture (6 operations → 1 line API key setting)
- Replace 10 lines of monkey patching boilerplate with 1-line inject_transport helper
- Remove cargo-cult error handling that allowed test to pass with API failures
- Create reusable transport_helpers.py for HTTP transport injection patterns
- Fix provider registration state pollution between batch test runs
- Test now works reliably in both individual and batch execution modes
The test is significantly cleaner and addresses root cause (provider registration timing)
rather than symptoms (cache clearing).
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add verification that o3-pro model was actually used (not just requested)
- Verify model_used and provider_used metadata fields are populated
- Add graceful handling for error responses in test
- Improve test documentation explaining what's being verified
- Confirm response parsing uses output_text field correctly
This ensures the test properly validates both that:
1. The o3-pro model was selected and used via the /v1/responses endpoint
2. The response metadata correctly identifies the model and provider
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Consolidates duplicated image validation logic from individual providers
into a reusable base class method. This improves maintainability and
ensures consistent validation across all providers.
- Added validate_image() method to ModelProvider base class
- Supports both file paths and data URLs
- Validates image format, size, and MIME types
- Added DEFAULT_MAX_IMAGE_SIZE_MB class constant (20MB)
- Refactored Gemini and OpenAI providers to use base validation
- Added comprehensive test suite with 19 tests
- Used minimal mocking approach with concrete test provider class
- Fix lint errors: trailing whitespace and deprecated typing imports
- Update test mock for o3-pro response format (output.content[] → output_text)
- Implement robust test isolation with monkeypatch fixture
- Clear provider registry cache to prevent test interference
- Ensure o3-pro tests pass in both individual and full suite execution
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add o3_pro_basic_math.json cassette for test_o3_pro_output_text_fix.py
- Remove unused o3_pro_content_capture.json cassette
- This allows tests to run without API keys in CI/CD
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
