Quick test mode for simulation tests

This commit is contained in:
Fahad
2025-06-23 17:53:03 +04:00
parent 14eaf930ed
commit 9167e6d845
2 changed files with 61 additions and 8 deletions

View File

@@ -128,7 +128,26 @@ python communication_simulator_test.py
python communication_simulator_test.py --verbose
```
#### Run Individual Simulator Tests (Recommended)
#### Quick Test Mode (Recommended for Time-Limited Testing)
```bash
# Run quick test mode - 6 essential tests that provide maximum functionality coverage
python communication_simulator_test.py --quick
# Run quick test mode with verbose output
python communication_simulator_test.py --quick --verbose
```
**Quick mode runs these 6 essential tests:**
- `cross_tool_continuation` - Cross-tool conversation memory testing (chat, thinkdeep, codereview, analyze, debug)
- `conversation_chain_validation` - Core conversation threading and memory validation
- `consensus_workflow_accurate` - Consensus tool with flash model and stance testing
- `codereview_validation` - CodeReview tool with flash model and multi-step workflows
- `planner_validation` - Planner tool with flash model and complex planning workflows
- `token_allocation_validation` - Token allocation and conversation history buildup testing
**Why these 6 tests:** They cover all major tools (chat, planner, consensus, codereview + analyze, debug, thinkdeep), extensively test conversation memory functionality, use flash/flashlite models, and provide comprehensive app functionality coverage in minimal time.
#### Run Individual Simulator Tests (For Detailed Testing)
```bash
# List all available tests
python communication_simulator_test.py --list-tests
@@ -223,15 +242,17 @@ python -m pytest tests/ -v
#### After Making Changes
1. Run quality checks again: `./code_quality_checks.sh`
2. Run integration tests locally: `./run_integration_tests.sh`
3. Run relevant simulator tests: `python communication_simulator_test.py --individual <test_name>`
4. Check logs for any issues: `tail -n 100 logs/mcp_server.log`
5. Restart Claude session to use updated code
3. Run quick test mode for fast validation: `python communication_simulator_test.py --quick`
4. Run relevant specific simulator tests if needed: `python communication_simulator_test.py --individual <test_name>`
5. Check logs for any issues: `tail -n 100 logs/mcp_server.log`
6. Restart Claude session to use updated code
#### Before Committing/PR
1. Final quality check: `./code_quality_checks.sh`
2. Run integration tests: `./run_integration_tests.sh`
3. Run full simulator test suite: `./run_integration_tests.sh --with-simulator`
4. Verify all tests pass 100%
3. Run quick test mode: `python communication_simulator_test.py --quick`
4. Run full simulator test suite (optional): `./run_integration_tests.sh --with-simulator`
5. Verify all tests pass 100%
### Common Troubleshooting
@@ -250,6 +271,9 @@ which python
#### Test Failures
```bash
# First try quick test mode to see if it's a general issue
python communication_simulator_test.py --quick --verbose
# Run individual failing test with verbose output
python communication_simulator_test.py --individual <test_name> --verbose

View File

@@ -38,6 +38,15 @@ Available tests:
debug_validation - Debug tool validation with actual bugs
conversation_chain_validation - Conversation chain continuity validation
Quick Test Mode (for time-limited testing):
Use --quick to run the essential 6 tests that provide maximum coverage:
- cross_tool_continuation
- conversation_chain_validation
- consensus_workflow_accurate
- codereview_validation
- planner_validation
- token_allocation_validation
Examples:
# Run all tests
python communication_simulator_test.py
@@ -48,6 +57,9 @@ Examples:
# Run a single test individually (with full standalone setup)
python communication_simulator_test.py --individual content_validation
# Run quick test mode (essential 6 tests for time-limited testing)
python communication_simulator_test.py --quick
# Force setup standalone server environment before running tests
python communication_simulator_test.py --setup
@@ -68,12 +80,13 @@ class CommunicationSimulator:
"""Simulates real-world Claude CLI communication with MCP Gemini server"""
def __init__(
self, verbose: bool = False, keep_logs: bool = False, selected_tests: list[str] = None, setup: bool = False
self, verbose: bool = False, keep_logs: bool = False, selected_tests: list[str] = None, setup: bool = False, quick_mode: bool = False
):
self.verbose = verbose
self.keep_logs = keep_logs
self.selected_tests = selected_tests or []
self.setup = setup
self.quick_mode = quick_mode
self.temp_dir = None
self.server_process = None
self.python_path = self._get_python_path()
@@ -83,6 +96,21 @@ class CommunicationSimulator:
self.test_registry = TEST_REGISTRY
# Define quick mode tests (essential tests for time-limited testing)
self.quick_mode_tests = [
"cross_tool_continuation",
"conversation_chain_validation",
"consensus_workflow_accurate",
"codereview_validation",
"planner_validation",
"token_allocation_validation"
]
# If quick mode is enabled, override selected_tests
if self.quick_mode:
self.selected_tests = self.quick_mode_tests
self.logger.info(f"Quick mode enabled - running {len(self.quick_mode_tests)} essential tests")
# Available test methods mapping
self.available_tests = {
name: self._create_test_runner(test_class) for name, test_class in self.test_registry.items()
@@ -415,6 +443,7 @@ def parse_arguments():
parser.add_argument("--tests", "-t", nargs="+", help="Specific tests to run (space-separated)")
parser.add_argument("--list-tests", action="store_true", help="List available tests and exit")
parser.add_argument("--individual", "-i", help="Run a single test individually")
parser.add_argument("--quick", "-q", action="store_true", help="Run quick test mode (6 essential tests for time-limited testing)")
parser.add_argument(
"--setup", action="store_true", help="Force setup standalone server environment using run-server.sh"
)
@@ -492,7 +521,7 @@ def main():
# Initialize simulator consistently for all use cases
simulator = CommunicationSimulator(
verbose=args.verbose, keep_logs=args.keep_logs, selected_tests=args.tests, setup=args.setup
verbose=args.verbose, keep_logs=args.keep_logs, selected_tests=args.tests, setup=args.setup, quick_mode=args.quick
)
# Determine execution mode and run