Add secaudit tool for security auditing (#117)
* WIP - working version * Implement required methods
This commit is contained in:
committed by
GitHub
parent
327c801c9b
commit
000d12dc3a
26
README.md
26
README.md
@@ -65,7 +65,8 @@ Because these AI models [clearly aren't when they get chatty →](docs/ai_banter
|
||||
- [`refactor`](#9-refactor---intelligent-code-refactoring) - Code refactoring with decomposition focus
|
||||
- [`tracer`](#10-tracer---static-code-analysis-prompt-generator) - Call-flow mapping and dependency tracing
|
||||
- [`testgen`](#11-testgen---comprehensive-test-generation) - Test generation with edge cases
|
||||
- [`docgen`](#12-docgen---comprehensive-documentation-generation) - Documentation generation with complexity analysis
|
||||
- [`secaudit`](#12-secaudit---comprehensive-security-audit) - Security audit with OWASP analysis
|
||||
- [`docgen`](#13-docgen---comprehensive-documentation-generation) - Documentation generation with complexity analysis
|
||||
|
||||
- **Advanced Usage**
|
||||
- [Advanced Features](#advanced-features) - AI-to-AI conversations, large prompts, web search
|
||||
@@ -247,6 +248,7 @@ and feel the difference.
|
||||
- **Code needs refactoring?** → `refactor` (intelligent refactoring with decomposition focus)
|
||||
- **Need call-flow analysis?** → `tracer` (generates prompts for execution tracing and dependency mapping)
|
||||
- **Need comprehensive tests?** → `testgen` (generates test suites with edge cases)
|
||||
- **Security concerns?** → `secaudit` (OWASP analysis, compliance evaluation, vulnerability assessment)
|
||||
- **Code needs documentation?** → `docgen` (generates comprehensive documentation with complexity analysis)
|
||||
- **Which models are available?** → `listmodels` (shows all configured providers and models)
|
||||
- **Server info?** → `version` (version and configuration details)
|
||||
@@ -274,9 +276,10 @@ and feel the difference.
|
||||
9. [`refactor`](docs/tools/refactor.md) - Code refactoring with decomposition focus
|
||||
10. [`tracer`](docs/tools/tracer.md) - Static code analysis prompt generator for call-flow mapping
|
||||
11. [`testgen`](docs/tools/testgen.md) - Comprehensive test generation with edge case coverage
|
||||
12. [`docgen`](docs/tools/docgen.md) - Comprehensive documentation generation with complexity analysis
|
||||
13. [`listmodels`](docs/tools/listmodels.md) - Display all available AI models organized by provider
|
||||
14. [`version`](docs/tools/version.md) - Get server version and configuration
|
||||
12. [`secaudit`](docs/tools/secaudit.md) - Comprehensive security audit with OWASP Top 10 analysis
|
||||
13. [`docgen`](docs/tools/docgen.md) - Comprehensive documentation generation with complexity analysis
|
||||
14. [`listmodels`](docs/tools/listmodels.md) - Display all available AI models organized by provider
|
||||
15. [`version`](docs/tools/version.md) - Get server version and configuration
|
||||
|
||||
### 1. `chat` - General Development Chat & Collaborative Thinking
|
||||
Your thinking partner for brainstorming, getting second opinions, and validating approaches. Perfect for technology comparisons, architecture discussions, and collaborative problem-solving.
|
||||
@@ -430,7 +433,16 @@ Use zen to generate tests for User.login() method
|
||||
|
||||
**[📖 Read More](docs/tools/testgen.md)** - Workflow-based test generation with comprehensive coverage
|
||||
|
||||
### 12. `docgen` - Comprehensive Documentation Generation
|
||||
### 12. `secaudit` - Comprehensive Security Audit
|
||||
Systematic OWASP-based security assessment with compliance evaluation. This workflow tool guides Claude through methodical security investigation steps with forced pauses between each step to ensure thorough vulnerability assessment, security pattern analysis, and compliance verification before providing expert analysis.
|
||||
|
||||
```
|
||||
Perform a secaudit with o3 on this e-commerce web application focusing on payment processing security and PCI DSS compliance
|
||||
```
|
||||
|
||||
**[📖 Read More](docs/tools/secaudit.md)** - OWASP Top 10 analysis with compliance framework support
|
||||
|
||||
### 13. `docgen` - Comprehensive Documentation Generation
|
||||
Generates thorough documentation with complexity analysis and gotcha identification. This workflow tool guides Claude through systematic investigation of code structure, function complexity, and documentation needs across multiple steps before generating comprehensive documentation that includes algorithmic complexity, call flow information, and unexpected behaviors that developers should know about.
|
||||
|
||||
```
|
||||
@@ -443,7 +455,7 @@ Use docgen to add complexity analysis to all the new swift functions I added but
|
||||
|
||||
**[📖 Read More](docs/tools/docgen.md)** - Workflow-based documentation generation with gotcha detection
|
||||
|
||||
### 13. `listmodels` - List Available Models
|
||||
### 14. `listmodels` - List Available Models
|
||||
Display all available AI models organized by provider, showing capabilities, context windows, and configuration status.
|
||||
|
||||
```
|
||||
@@ -452,7 +464,7 @@ Use zen to list available models
|
||||
|
||||
**[📖 Read More](docs/tools/listmodels.md)** - Model capabilities and configuration details
|
||||
|
||||
### 14. `version` - Server Information
|
||||
### 15. `version` - Server Information
|
||||
Get server version, configuration details, and system status for debugging and troubleshooting.
|
||||
|
||||
```
|
||||
|
||||
222
docs/tools/secaudit.md
Normal file
222
docs/tools/secaudit.md
Normal file
@@ -0,0 +1,222 @@
|
||||
# Secaudit Tool - Comprehensive Security Audit
|
||||
|
||||
**Systematic OWASP-based security assessment with compliance evaluation through workflow-driven investigation**
|
||||
|
||||
The `secaudit` tool provides comprehensive security auditing capabilities with systematic OWASP Top 10 assessment, compliance framework evaluation,
|
||||
and threat modeling. This workflow tool guides Claude through methodical security investigation steps with forced pauses between each step to ensure
|
||||
thorough vulnerability assessment, security pattern analysis, and compliance verification before providing expert analysis.
|
||||
|
||||
**Important**: AI models may not identify all security vulnerabilities. Always perform additional manual security reviews,
|
||||
penetration testing, and verification.
|
||||
|
||||
## How the Workflow Works
|
||||
|
||||
The secaudit tool implements a **structured 6-step security workflow** that ensures comprehensive security assessment:
|
||||
|
||||
**Investigation Phase (Claude-Led):**
|
||||
1. **Step 1**: Security Scope Analysis - Claude identifies application type, tech stack, attack surface, and compliance requirements
|
||||
2. **Step 2**: Authentication & Authorization Assessment - Analyzes auth mechanisms, session management, and access controls
|
||||
3. **Step 3**: Input Validation & Data Security - Reviews input handling, data protection, and injection vulnerabilities
|
||||
4. **Step 4**: OWASP Top 10 (2021) Review - Systematic assessment of all OWASP categories with specific findings
|
||||
5. **Step 5**: Dependencies & Infrastructure - Security analysis of third-party components and deployment configurations
|
||||
6. **Step 6**: Compliance & Risk Assessment - Evaluation against specified compliance frameworks and risk prioritization
|
||||
|
||||
**Expert Analysis Phase:**
|
||||
After Claude completes the investigation (unless confidence is **certain**):
|
||||
- Complete security assessment summary with all vulnerabilities and evidence
|
||||
- OWASP Top 10 systematic findings with severity classifications
|
||||
- Compliance framework gap analysis and remediation recommendations
|
||||
- Risk-prioritized remediation roadmap based on threat level and business impact
|
||||
|
||||
**Special Note**: If you want Claude to perform the entire security audit without calling another model, you can include "don't use any other model" in your prompt, and Claude will complete the full workflow independently.
|
||||
|
||||
## Model Recommendation
|
||||
|
||||
This tool particularly benefits from Gemini Pro or O3 models due to their advanced reasoning capabilities and large context windows, which allow comprehensive security analysis across complex codebases. Security audits require understanding subtle attack vectors and cross-component interactions that benefit from deeper analytical capabilities.
|
||||
|
||||
## Example Prompts
|
||||
|
||||
```
|
||||
Perform a secaudit with o3 on this e-commerce web application focusing on payment processing security and PCI DSS compliance
|
||||
```
|
||||
|
||||
```
|
||||
Use secaudit to conduct a comprehensive security audit of the authentication system, threat level high, focus on enterprise
|
||||
security patterns and HIPAA compliance
|
||||
```
|
||||
|
||||
## Pro Tip: Multi-Scope Security Assessment
|
||||
|
||||
**You can run parallel security audits for different application components:**
|
||||
|
||||
```
|
||||
Start separate sub-tasks, in one start a secaudit for critical payment processing components focusing on PCI DSS with gemini pro,
|
||||
and in the other for user management focusing on OWASP authentication vulnerabilities with o4-mini, then combine into a unified
|
||||
security remediation plan using planner
|
||||
```
|
||||
|
||||
## Key Features
|
||||
|
||||
- **OWASP Top 10 (2021) systematic assessment** with specific vulnerability identification
|
||||
- **Multi-compliance framework support**: SOC2, PCI DSS, HIPAA, GDPR, FedRAMP
|
||||
- **Threat-level aware analysis**: Critical, high, medium, low threat classifications
|
||||
- **Technology-specific security patterns**: Web apps, APIs, mobile, cloud, enterprise systems
|
||||
- **Risk-based prioritization**: Business impact and exploitability assessment
|
||||
- **Audit focus customization**: Comprehensive, authentication, data protection, infrastructure
|
||||
- **Image support**: Security analysis from architecture diagrams, network topology, or security findings
|
||||
- **Multi-file security analysis**: Cross-component vulnerability identification
|
||||
- **Compliance gap analysis**: Specific framework requirements with remediation guidance
|
||||
- **Attack surface mapping**: Entry points, data flows, and privilege boundaries
|
||||
- **Security control effectiveness**: Evaluation of existing security measures
|
||||
|
||||
## Tool Parameters
|
||||
|
||||
**Workflow Investigation Parameters (used during step-by-step process):**
|
||||
- `step`: Current security investigation step description (required for each step)
|
||||
- `step_number`: Current step number in audit sequence (required)
|
||||
- `total_steps`: Estimated total investigation steps (typically 4-6, adjustable)
|
||||
- `next_step_required`: Whether another investigation step is needed
|
||||
- `findings`: Security discoveries and evidence collected in this step (required)
|
||||
- `files_checked`: All files examined during security investigation
|
||||
- `relevant_files`: Files directly relevant to security assessment (required in step 1)
|
||||
- `relevant_context`: Methods/functions/classes central to security findings
|
||||
- `issues_found`: Security issues identified with severity levels
|
||||
- `confidence`: Confidence level in security assessment completeness (exploring/low/medium/high/certain)
|
||||
- `backtrack_from_step`: Step number to backtrack from (for revisions)
|
||||
- `images`: Architecture diagrams, security documentation, or visual references
|
||||
|
||||
**Initial Security Configuration (used in step 1):**
|
||||
- `model`: auto|pro|flash|o3|o3-mini|o4-mini|o4-mini-high|gpt4.1 (default: server default)
|
||||
- `security_scope`: Application context, technology stack, and security boundary definition (required)
|
||||
- `threat_level`: low|medium|high|critical (default: medium) - determines assessment depth and urgency
|
||||
- `compliance_requirements`: List of compliance frameworks to assess against (e.g., ["PCI DSS", "SOC2"])
|
||||
- `audit_focus`: comprehensive|authentication|data_protection|infrastructure|api_security (default: comprehensive)
|
||||
- `severity_filter`: critical|high|medium|low|all (default: all)
|
||||
- `temperature`: Temperature for analytical consistency (0-1, default 0.2)
|
||||
- `thinking_mode`: minimal|low|medium|high|max (default: medium, Gemini only)
|
||||
- `use_websearch`: Enable web search for security best practices and vulnerability databases (default: true)
|
||||
- `use_assistant_model`: Whether to use expert security analysis phase (default: true)
|
||||
- `continuation_id`: Continue previous security audit discussions
|
||||
|
||||
## Audit Focus Areas
|
||||
|
||||
**Comprehensive (default):**
|
||||
- Full OWASP Top 10 assessment with all security domains
|
||||
- Authentication, authorization, data protection, infrastructure
|
||||
- Best for complete security posture evaluation
|
||||
|
||||
**Authentication:**
|
||||
- Focused on identity, access management, and session security
|
||||
- Multi-factor authentication, password policies, privilege escalation
|
||||
- Best for user management and access control systems
|
||||
|
||||
**Data Protection:**
|
||||
- Encryption, data handling, privacy controls, and compliance
|
||||
- Input validation, output encoding, data classification
|
||||
- Best for applications handling sensitive or regulated data
|
||||
|
||||
**Infrastructure:**
|
||||
- Deployment security, configuration management, dependency security
|
||||
- Network security, container security, cloud security posture
|
||||
- Best for DevOps and infrastructure security assessment
|
||||
|
||||
**API Security:**
|
||||
- REST/GraphQL security, rate limiting, API authentication
|
||||
- Input validation, authorization patterns, API gateway security
|
||||
- Best for API-first applications and microservices
|
||||
|
||||
## Threat Levels
|
||||
|
||||
Security assessment depth and urgency:
|
||||
|
||||
- **🔴 CRITICAL**: Mission-critical systems, high-value targets, regulatory requirements
|
||||
- **🟠 HIGH**: Business-critical applications, customer data handling, financial systems
|
||||
- **🟡 MEDIUM**: Standard business applications, internal tools, moderate risk exposure
|
||||
- **🟢 LOW**: Development environments, non-sensitive applications, proof-of-concepts
|
||||
|
||||
## Compliance Frameworks
|
||||
|
||||
Supported compliance assessments:
|
||||
|
||||
- **SOC2**: Security, availability, processing integrity, confidentiality, privacy
|
||||
- **PCI DSS**: Payment card industry data security standards
|
||||
- **HIPAA**: Healthcare information privacy and security
|
||||
- **GDPR**: General data protection regulation compliance
|
||||
- **FedRAMP**: Federal risk and authorization management program
|
||||
- **ISO27001**: Information security management systems
|
||||
- **NIST**: Cybersecurity framework controls
|
||||
|
||||
## OWASP Top 10 (2021) Coverage
|
||||
|
||||
Systematic assessment includes:
|
||||
|
||||
1. **A01 Broken Access Control**: Authorization flaws and privilege escalation
|
||||
2. **A02 Cryptographic Failures**: Encryption and data protection issues
|
||||
3. **A03 Injection**: SQL, NoSQL, OS, and LDAP injection vulnerabilities
|
||||
4. **A04 Insecure Design**: Security design flaws and threat modeling gaps
|
||||
5. **A05 Security Misconfiguration**: Configuration and hardening issues
|
||||
6. **A06 Vulnerable Components**: Third-party and dependency vulnerabilities
|
||||
7. **A07 Identification & Authentication Failures**: Authentication bypass and session management
|
||||
8. **A08 Software & Data Integrity Failures**: Supply chain and integrity violations
|
||||
9. **A09 Security Logging & Monitoring Failures**: Detection and response capabilities
|
||||
10. **A10 Server-Side Request Forgery**: SSRF and related vulnerabilities
|
||||
|
||||
## Usage Examples
|
||||
|
||||
**Comprehensive E-commerce Security Audit:**
|
||||
```
|
||||
"Conduct a comprehensive secaudit with gemini pro for our Node.js e-commerce platform, threat level high,
|
||||
compliance requirements PCI DSS and SOC2, focus on payment processing security"
|
||||
```
|
||||
|
||||
**Authentication System Security Review:**
|
||||
```
|
||||
"Use o3 to perform secaudit on authentication microservice, focus on authentication,
|
||||
threat level critical, check for OWASP A07 and multi-factor authentication implementation"
|
||||
```
|
||||
|
||||
**API Security Assessment:**
|
||||
```
|
||||
"Secaudit our REST API gateway with gemini pro, audit focus api_security,
|
||||
compliance requirements GDPR, threat level medium"
|
||||
```
|
||||
|
||||
**Infrastructure Security Review:**
|
||||
```
|
||||
"Perform secaudit on Kubernetes deployment manifests with o3, focus infrastructure,
|
||||
threat level high, include container security and network policies"
|
||||
```
|
||||
|
||||
**Quick Security Scan:**
|
||||
```
|
||||
"Fast secaudit of user registration flow with flash, focus authentication,
|
||||
severity filter critical and high only"
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
- **Define clear security scope**: Specify application type, tech stack, and security boundaries
|
||||
- **Set appropriate threat levels**: Match assessment depth to risk exposure and criticality
|
||||
- **Include compliance requirements**: Specify relevant frameworks for regulatory alignment
|
||||
- **Use parallel audits**: Run separate assessments for different components or compliance frameworks
|
||||
- **Provide architectural context**: Include system diagrams, data flow documentation, or deployment topology
|
||||
- **Focus audit scope**: Use audit_focus for targeted assessments of specific security domains
|
||||
- **Follow up on findings**: Use continuation feature to dive deeper into specific vulnerabilities
|
||||
|
||||
## Output Format
|
||||
|
||||
Security audits include:
|
||||
- **Executive Security Summary**: Overall security posture and critical findings
|
||||
- **OWASP Top 10 Assessment**: Systematic review of each category with specific findings
|
||||
- **Compliance Gap Analysis**: Framework-specific requirements and current compliance status
|
||||
- **Risk-Prioritized Findings**: Vulnerabilities ordered by exploitability and business impact
|
||||
- **Remediation Roadmap**: Phased approach to security improvements with quick wins
|
||||
- **Security Architecture Recommendations**: Structural improvements for long-term security posture
|
||||
|
||||
## When to Use Secaudit vs Other Tools
|
||||
|
||||
- **Use `secaudit`** for: Comprehensive security assessment, compliance evaluation, OWASP-based vulnerability analysis
|
||||
- **Use `codereview`** for: General code quality with some security considerations
|
||||
- **Use `analyze`** for: Understanding security architecture without vulnerability assessment
|
||||
- **Use `debug`** for: Investigating specific security incidents or exploit attempts
|
||||
- **Use `precommit`** for: Pre-deployment security validation and change impact assessment
|
||||
@@ -67,6 +67,7 @@ from tools import ( # noqa: E402
|
||||
PlannerTool,
|
||||
PrecommitTool,
|
||||
RefactorTool,
|
||||
SecauditTool,
|
||||
TestGenTool,
|
||||
ThinkDeepTool,
|
||||
TracerTool,
|
||||
@@ -170,6 +171,7 @@ TOOLS = {
|
||||
"codereview": CodeReviewTool(), # Comprehensive step-by-step code review workflow with expert analysis
|
||||
"precommit": PrecommitTool(), # Step-by-step pre-commit validation workflow
|
||||
"debug": DebugIssueTool(), # Root cause analysis and debugging assistance
|
||||
"secaudit": SecauditTool(), # Comprehensive security audit with OWASP Top 10 and compliance coverage
|
||||
"docgen": DocgenTool(), # Step-by-step documentation generation with complexity analysis
|
||||
"analyze": AnalyzeTool(), # General-purpose file and code analysis
|
||||
"refactor": RefactorTool(), # Step-by-step refactoring analysis workflow with expert validation
|
||||
@@ -216,6 +218,11 @@ PROMPT_TEMPLATES = {
|
||||
"description": "Debug an issue or error",
|
||||
"template": "Help debug this issue with {model}",
|
||||
},
|
||||
"secaudit": {
|
||||
"name": "secaudit",
|
||||
"description": "Comprehensive security audit with OWASP Top 10 coverage",
|
||||
"template": "Perform comprehensive security audit with {model}",
|
||||
},
|
||||
"docgen": {
|
||||
"name": "docgen",
|
||||
"description": "Generate comprehensive code documentation with complexity analysis",
|
||||
|
||||
@@ -35,6 +35,7 @@ from .test_prompt_size_limit_bug import PromptSizeLimitBugTest
|
||||
|
||||
# Redis validation test removed - no longer needed for standalone server
|
||||
from .test_refactor_validation import RefactorValidationTest
|
||||
from .test_secaudit_validation import SecauditValidationTest
|
||||
from .test_testgen_validation import TestGenValidationTest
|
||||
from .test_thinkdeep_validation import ThinkDeepWorkflowValidationTest
|
||||
from .test_token_allocation_validation import TokenAllocationValidationTest
|
||||
@@ -65,6 +66,7 @@ TEST_REGISTRY = {
|
||||
"testgen_validation": TestGenValidationTest,
|
||||
"thinkdeep_validation": ThinkDeepWorkflowValidationTest,
|
||||
"refactor_validation": RefactorValidationTest,
|
||||
"secaudit_validation": SecauditValidationTest,
|
||||
"debug_validation": DebugValidationTest,
|
||||
"debug_certain_confidence": DebugCertainConfidenceTest,
|
||||
"conversation_chain_validation": ConversationChainValidationTest,
|
||||
@@ -102,6 +104,7 @@ __all__ = [
|
||||
"TestGenValidationTest",
|
||||
"ThinkDeepWorkflowValidationTest",
|
||||
"RefactorValidationTest",
|
||||
"SecauditValidationTest",
|
||||
"DebugValidationTest",
|
||||
"DebugCertainConfidenceTest",
|
||||
"ConversationChainValidationTest",
|
||||
|
||||
610
simulator_tests/test_secaudit_validation.py
Normal file
610
simulator_tests/test_secaudit_validation.py
Normal file
@@ -0,0 +1,610 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
SECAUDIT Tool Validation Test
|
||||
|
||||
Tests the secaudit tool's capabilities using the workflow architecture.
|
||||
This validates that the workflow-based security audit provides step-by-step
|
||||
analysis with proper investigation guidance and expert analysis integration.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
from .conversation_base_test import ConversationBaseTest
|
||||
|
||||
|
||||
class SecauditValidationTest(ConversationBaseTest):
|
||||
"""Test secaudit tool with workflow architecture"""
|
||||
|
||||
@property
|
||||
def test_name(self) -> str:
|
||||
return "secaudit_validation"
|
||||
|
||||
@property
|
||||
def test_description(self) -> str:
|
||||
return "SECAUDIT tool validation with security audit workflow architecture"
|
||||
|
||||
def run_test(self) -> bool:
|
||||
"""Test secaudit tool capabilities"""
|
||||
# Set up the test environment
|
||||
self.setUp()
|
||||
|
||||
try:
|
||||
self.logger.info("Test: SECAUDIT tool validation (security workflow architecture)")
|
||||
|
||||
# Create test code with various security vulnerabilities
|
||||
self._create_test_code_for_audit()
|
||||
|
||||
# Test 1: Single audit session with multiple steps
|
||||
if not self._test_single_audit_session():
|
||||
return False
|
||||
|
||||
# Test 2: Audit with specific focus areas
|
||||
if not self._test_focused_security_audit():
|
||||
return False
|
||||
|
||||
# Test 3: Complete audit with expert analysis using fast model
|
||||
if not self._test_complete_audit_with_analysis():
|
||||
return False
|
||||
|
||||
# Test 4: Certain confidence behavior
|
||||
if not self._test_certain_confidence():
|
||||
return False
|
||||
|
||||
# Test 5: Continuation test with chat tool
|
||||
if not self._test_continuation_with_chat():
|
||||
return False
|
||||
|
||||
# Test 6: Model selection control
|
||||
if not self._test_model_selection():
|
||||
return False
|
||||
|
||||
self.logger.info(" ✅ All secaudit validation tests passed")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"SECAUDIT validation test failed: {e}")
|
||||
return False
|
||||
|
||||
def _create_test_code_for_audit(self):
|
||||
"""Create test files with various security vulnerabilities"""
|
||||
# Create an authentication module with multiple security issues
|
||||
auth_code = """#!/usr/bin/env python3
|
||||
import hashlib
|
||||
import pickle
|
||||
import sqlite3
|
||||
from flask import request, session
|
||||
|
||||
class AuthenticationManager:
|
||||
def __init__(self, db_path="users.db"):
|
||||
# A01: Broken Access Control - No proper session management
|
||||
self.db_path = db_path
|
||||
self.sessions = {} # In-memory session storage
|
||||
def login(self, username, password):
|
||||
'''User login with various security vulnerabilities'''
|
||||
# A03: Injection - SQL injection vulnerability
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Direct string interpolation in SQL query
|
||||
query = f"SELECT id, password_hash FROM users WHERE username = '{username}'"
|
||||
cursor.execute(query)
|
||||
|
||||
user = cursor.fetchone()
|
||||
if not user:
|
||||
return {"status": "failed", "message": "User not found"}
|
||||
|
||||
# A02: Cryptographic Failures - Weak hashing algorithm
|
||||
password_hash = hashlib.md5(password.encode()).hexdigest()
|
||||
|
||||
if user[1] == password_hash:
|
||||
# A07: Identification and Authentication Failures - Weak session generation
|
||||
session_id = hashlib.md5(f"{username}{password}".encode()).hexdigest()
|
||||
self.sessions[session_id] = {"user_id": user[0], "username": username}
|
||||
|
||||
return {"status": "success", "session_id": session_id}
|
||||
else:
|
||||
return {"status": "failed", "message": "Invalid password"}
|
||||
|
||||
def reset_password(self, email):
|
||||
'''Password reset with security issues'''
|
||||
# A04: Insecure Design - No rate limiting or validation
|
||||
reset_token = hashlib.md5(email.encode()).hexdigest()
|
||||
|
||||
# A09: Security Logging and Monitoring Failures - No security event logging
|
||||
# Simply returns token without any verification or logging
|
||||
return {"reset_token": reset_token, "url": f"/reset?token={reset_token}"}
|
||||
|
||||
def deserialize_user_data(self, data):
|
||||
'''Unsafe deserialization'''
|
||||
# A08: Software and Data Integrity Failures - Insecure deserialization
|
||||
return pickle.loads(data)
|
||||
|
||||
def get_user_profile(self, user_id):
|
||||
'''Get user profile with authorization issues'''
|
||||
# A01: Broken Access Control - No authorization check
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Fetches any user profile without checking permissions
|
||||
cursor.execute("SELECT * FROM users WHERE id = ?", (user_id,))
|
||||
return cursor.fetchone()
|
||||
"""
|
||||
|
||||
# Create authentication file
|
||||
self.auth_file = self.create_additional_test_file("auth_manager.py", auth_code)
|
||||
self.logger.info(f" ✅ Created authentication file with security issues: {self.auth_file}")
|
||||
|
||||
# Create API endpoint with additional vulnerabilities
|
||||
api_code = """#!/usr/bin/env python3
|
||||
from flask import Flask, request, jsonify
|
||||
import os
|
||||
import subprocess
|
||||
import requests
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
# A05: Security Misconfiguration - Debug mode enabled
|
||||
app.config['DEBUG'] = True
|
||||
app.config['SECRET_KEY'] = 'dev-secret-key' # Hardcoded secret
|
||||
|
||||
@app.route('/api/search', methods=['GET'])
|
||||
def search():
|
||||
'''Search endpoint with multiple vulnerabilities'''
|
||||
# A03: Injection - XSS vulnerability, no input sanitization
|
||||
query = request.args.get('q', '')
|
||||
|
||||
# A03: Injection - Command injection vulnerability
|
||||
if 'file:' in query:
|
||||
filename = query.split('file:')[1]
|
||||
# Direct command execution
|
||||
result = subprocess.run(f"cat {filename}", shell=True, capture_output=True, text=True)
|
||||
return jsonify({"result": result.stdout})
|
||||
|
||||
# A10: Server-Side Request Forgery (SSRF)
|
||||
if query.startswith('http'):
|
||||
# No validation of URL, allows internal network access
|
||||
response = requests.get(query)
|
||||
return jsonify({"content": response.text})
|
||||
|
||||
# Return search results without output encoding
|
||||
return f"<h1>Search Results for: {query}</h1>"
|
||||
|
||||
@app.route('/api/admin', methods=['GET'])
|
||||
def admin_panel():
|
||||
'''Admin panel with broken access control'''
|
||||
# A01: Broken Access Control - No authentication check
|
||||
# Anyone can access admin functionality
|
||||
action = request.args.get('action')
|
||||
|
||||
if action == 'delete_user':
|
||||
user_id = request.args.get('user_id')
|
||||
# Performs privileged action without authorization
|
||||
return jsonify({"status": "User deleted", "user_id": user_id})
|
||||
|
||||
return jsonify({"status": "Admin panel"})
|
||||
|
||||
@app.route('/api/upload', methods=['POST'])
|
||||
def upload_file():
|
||||
'''File upload with security issues'''
|
||||
# A05: Security Misconfiguration - No file type validation
|
||||
file = request.files.get('file')
|
||||
if file:
|
||||
# Saves any file type to server
|
||||
filename = file.filename
|
||||
file.save(os.path.join('/tmp', filename))
|
||||
|
||||
# A03: Path traversal vulnerability
|
||||
return jsonify({"status": "File uploaded", "path": f"/tmp/{filename}"})
|
||||
|
||||
return jsonify({"error": "No file provided"})
|
||||
|
||||
# A06: Vulnerable and Outdated Components
|
||||
# Using old Flask version with known vulnerabilities (hypothetical)
|
||||
# requirements.txt: Flask==0.12.2 (known security issues)
|
||||
|
||||
if __name__ == '__main__':
|
||||
# A05: Security Misconfiguration - Running on all interfaces
|
||||
app.run(host='0.0.0.0', port=5000, debug=True)
|
||||
"""
|
||||
|
||||
# Create API file
|
||||
self.api_file = self.create_additional_test_file("api_endpoints.py", api_code)
|
||||
self.logger.info(f" ✅ Created API file with security vulnerabilities: {self.api_file}")
|
||||
|
||||
def _test_single_audit_session(self) -> bool:
|
||||
"""Test a single security audit session with multiple steps"""
|
||||
self.logger.info(" 🔧 Testing single audit session...")
|
||||
|
||||
try:
|
||||
# Step 1: Initial security audit request
|
||||
response, continuation_id = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": f"Begin security audit of authentication system in {self.auth_file}",
|
||||
"step_number": 1,
|
||||
"total_steps": 6,
|
||||
"next_step_required": True,
|
||||
"findings": "Starting security assessment",
|
||||
"relevant_files": [self.auth_file],
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if not response:
|
||||
self.logger.error("Failed to call secaudit tool")
|
||||
return False
|
||||
|
||||
# Parse and validate the response
|
||||
try:
|
||||
response_data = json.loads(response) if response else {}
|
||||
except json.JSONDecodeError:
|
||||
response_data = {}
|
||||
|
||||
# Check if it's asking for investigation
|
||||
status = response_data.get("status", "")
|
||||
if status != "pause_for_secaudit":
|
||||
self.logger.error(f"Expected pause_for_secaudit status, got: {status}")
|
||||
return False
|
||||
|
||||
# Step 2: Continue with findings
|
||||
response2, _ = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": "Examined authentication module and found critical security vulnerabilities",
|
||||
"step_number": 2,
|
||||
"total_steps": 6,
|
||||
"next_step_required": True,
|
||||
"findings": (
|
||||
"Found multiple OWASP Top 10 vulnerabilities: "
|
||||
"1. SQL injection in login method (line 88) - direct string interpolation in query "
|
||||
"2. Weak MD5 hashing for passwords (line 96) - cryptographically broken "
|
||||
"3. Insecure session management (line 100) - predictable session IDs "
|
||||
"4. Unsafe deserialization (line 119) - pickle.loads without validation"
|
||||
),
|
||||
"files_checked": [self.auth_file],
|
||||
"relevant_files": [self.auth_file],
|
||||
"relevant_context": ["AuthenticationManager.login", "AuthenticationManager.deserialize_user_data"],
|
||||
"issues_found": [
|
||||
{"severity": "critical", "description": "SQL injection vulnerability in login method"},
|
||||
{"severity": "high", "description": "Weak MD5 password hashing"},
|
||||
{"severity": "high", "description": "Insecure session management"},
|
||||
{"severity": "critical", "description": "Unsafe deserialization vulnerability"},
|
||||
],
|
||||
"confidence": "medium",
|
||||
"continuation_id": continuation_id,
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if not response2:
|
||||
self.logger.error("Failed to continue to step 2")
|
||||
return False
|
||||
|
||||
self.logger.info(" ✅ Single audit session test passed")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Single audit session test failed: {e}")
|
||||
return False
|
||||
|
||||
def _test_focused_security_audit(self) -> bool:
|
||||
"""Test security audit with specific focus areas"""
|
||||
self.logger.info(" 🔧 Testing focused security audit...")
|
||||
|
||||
try:
|
||||
# Request OWASP-focused audit
|
||||
response, continuation_id = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": f"Begin OWASP-focused security audit of {self.api_file}",
|
||||
"step_number": 1,
|
||||
"total_steps": 4,
|
||||
"next_step_required": True,
|
||||
"findings": "Starting OWASP Top 10 focused security assessment",
|
||||
"relevant_files": [self.api_file],
|
||||
"security_scope": "Web API endpoints",
|
||||
"threat_level": "high",
|
||||
"audit_focus": "owasp",
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if not response:
|
||||
self.logger.error("Failed to start OWASP-focused audit")
|
||||
return False
|
||||
|
||||
# Verify the audit was configured correctly
|
||||
try:
|
||||
response_data = json.loads(response)
|
||||
# The tool should acknowledge the OWASP focus
|
||||
if response_data.get("status") == "pause_for_secaudit":
|
||||
self.logger.info(" ✅ Focused security audit test passed")
|
||||
return True
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
self.logger.error("Expected proper OWASP-focused configuration")
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Focused security audit test failed: {e}")
|
||||
return False
|
||||
|
||||
def _test_complete_audit_with_analysis(self) -> bool:
|
||||
"""Test complete security audit with expert analysis"""
|
||||
self.logger.info(" 🔧 Testing complete audit with expert analysis...")
|
||||
|
||||
try:
|
||||
# Step 1: Start fresh audit
|
||||
response1, continuation_id = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": f"Begin comprehensive security audit of {self.auth_file} and {self.api_file}",
|
||||
"step_number": 1,
|
||||
"total_steps": 3,
|
||||
"next_step_required": True,
|
||||
"findings": "Starting OWASP Top 10 security assessment of authentication and API modules",
|
||||
"relevant_files": [self.auth_file, self.api_file],
|
||||
"security_scope": "Web application with authentication and API endpoints",
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if not response1:
|
||||
self.logger.error("Failed to start comprehensive audit")
|
||||
return False
|
||||
|
||||
# Step 2: Continue with detailed findings
|
||||
response2, _ = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": "Completed comprehensive security investigation of both modules",
|
||||
"step_number": 2,
|
||||
"total_steps": 3,
|
||||
"next_step_required": True,
|
||||
"findings": (
|
||||
"Found critical OWASP vulnerabilities across both modules: "
|
||||
"A01: Broken Access Control in admin panel, "
|
||||
"A03: SQL injection in login and command injection in search, "
|
||||
"A02: Weak cryptography with MD5 hashing, "
|
||||
"A05: Security misconfiguration with debug mode enabled, "
|
||||
"A07: Weak session management, "
|
||||
"A08: Insecure deserialization, "
|
||||
"A10: SSRF vulnerability in search endpoint"
|
||||
),
|
||||
"files_checked": [self.auth_file, self.api_file],
|
||||
"relevant_files": [self.auth_file, self.api_file],
|
||||
"relevant_context": [
|
||||
"AuthenticationManager.login",
|
||||
"AuthenticationManager.deserialize_user_data",
|
||||
"api.search",
|
||||
"api.admin_panel",
|
||||
],
|
||||
"issues_found": [
|
||||
{"severity": "critical", "description": "SQL injection in login method"},
|
||||
{"severity": "critical", "description": "Command injection in search endpoint"},
|
||||
{"severity": "critical", "description": "SSRF vulnerability allowing internal network access"},
|
||||
{"severity": "high", "description": "Broken access control on admin panel"},
|
||||
{"severity": "high", "description": "Insecure deserialization vulnerability"},
|
||||
{"severity": "high", "description": "XSS vulnerability in search results"},
|
||||
{"severity": "medium", "description": "Weak MD5 password hashing"},
|
||||
{"severity": "medium", "description": "Security misconfiguration - debug mode enabled"},
|
||||
],
|
||||
"confidence": "high",
|
||||
"continuation_id": continuation_id,
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
# Final step - skip expert analysis to avoid timeout
|
||||
response3, _ = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": "Complete security assessment with all vulnerabilities documented",
|
||||
"step_number": 3,
|
||||
"total_steps": 3,
|
||||
"next_step_required": False,
|
||||
"findings": "Security audit complete with 8 vulnerabilities identified across OWASP categories",
|
||||
"files_checked": [self.auth_file, self.api_file],
|
||||
"relevant_files": [self.auth_file, self.api_file],
|
||||
"confidence": "high", # High confidence to trigger expert analysis
|
||||
"continuation_id": continuation_id,
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if response3:
|
||||
# Check for expert analysis or completion status
|
||||
try:
|
||||
response_data = json.loads(response3)
|
||||
status = response_data.get("status", "")
|
||||
# Either expert analysis completed or security analysis complete
|
||||
if status in ["complete", "security_analysis_complete"]:
|
||||
self.logger.info(" ✅ Complete audit with expert analysis test passed")
|
||||
return True
|
||||
except json.JSONDecodeError:
|
||||
# If not JSON, check for security content (expert analysis output)
|
||||
if "security" in response3.lower() or "vulnerability" in response3.lower():
|
||||
self.logger.info(" ✅ Complete audit with expert analysis test passed")
|
||||
return True
|
||||
|
||||
self.logger.error("Expected expert security analysis or completion")
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Complete audit with analysis test failed: {e}")
|
||||
return False
|
||||
|
||||
def _test_certain_confidence(self) -> bool:
|
||||
"""Test behavior when confidence is certain"""
|
||||
self.logger.info(" 🔧 Testing certain confidence behavior...")
|
||||
|
||||
try:
|
||||
# Request with certain confidence
|
||||
response, _ = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": f"Security audit complete for {self.auth_file}",
|
||||
"step_number": 1,
|
||||
"total_steps": 1,
|
||||
"next_step_required": False,
|
||||
"findings": "Critical SQL injection vulnerability confirmed in login method",
|
||||
"files_checked": [self.auth_file],
|
||||
"relevant_files": [self.auth_file],
|
||||
"issues_found": [
|
||||
{"severity": "critical", "description": "SQL injection vulnerability in login method"}
|
||||
],
|
||||
"confidence": "certain",
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if not response:
|
||||
self.logger.error("Failed to execute certain confidence test")
|
||||
return False
|
||||
|
||||
try:
|
||||
response_data = json.loads(response)
|
||||
# With certain confidence, should complete without expert analysis
|
||||
if response_data.get("status") == "security_analysis_complete":
|
||||
self.logger.info(" ✅ Certain confidence correctly completes without expert analysis")
|
||||
return True
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
# Check if findings are shown directly
|
||||
response_lower = response.lower()
|
||||
if "sql injection" in response_lower or "vulnerability" in response_lower:
|
||||
self.logger.info(" ✅ Certain confidence shows findings directly")
|
||||
return True
|
||||
|
||||
self.logger.error("Expected completion or direct findings with certain confidence")
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Certain confidence test failed: {e}")
|
||||
return False
|
||||
|
||||
def _test_continuation_with_chat(self) -> bool:
|
||||
"""Test continuation functionality with chat tool"""
|
||||
self.logger.info(" 🔧 Testing continuation with chat tool...")
|
||||
|
||||
try:
|
||||
# First, run a security audit that generates a continuation_id
|
||||
response1, continuation_id = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": f"Start analyzing {self.auth_file} for authentication vulnerabilities",
|
||||
"step_number": 1,
|
||||
"total_steps": 4,
|
||||
"next_step_required": True,
|
||||
"findings": "Beginning authentication security analysis",
|
||||
"relevant_files": [self.auth_file],
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if not response1:
|
||||
self.logger.error("Failed to start audit for continuation test")
|
||||
return False
|
||||
|
||||
# Extract continuation_id if present
|
||||
if not continuation_id:
|
||||
self.logger.info(" ⚠️ No continuation_id returned, checking response")
|
||||
try:
|
||||
response_data = json.loads(response1)
|
||||
# Look for thread_id in metadata
|
||||
metadata = response_data.get("metadata", {})
|
||||
continuation_id = metadata.get("thread_id")
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
if continuation_id:
|
||||
# Now test using chat tool with continuation
|
||||
chat_response, _ = self.call_mcp_tool_direct(
|
||||
"chat",
|
||||
{
|
||||
"prompt": "Can you tell me more about the SQL injection vulnerability details found in the security audit?",
|
||||
"continuation_id": continuation_id,
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if chat_response:
|
||||
self.logger.info(" ✅ Chat tool continuation test passed")
|
||||
return True
|
||||
else:
|
||||
# Without continuation_id, just verify the audit step worked
|
||||
if response1:
|
||||
self.logger.info(" ✅ Audit step completed (continuation test limited)")
|
||||
return True
|
||||
|
||||
self.logger.error("Expected successful continuation or audit step")
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Continuation test failed: {e}")
|
||||
return False
|
||||
|
||||
def _test_model_selection(self) -> bool:
|
||||
"""Test model selection and skip expert analysis option"""
|
||||
self.logger.info(" 🔧 Testing model selection control...")
|
||||
|
||||
try:
|
||||
# Test 1: Explicit model selection
|
||||
response1, _ = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": f"Analyze {self.api_file} for SSRF vulnerabilities",
|
||||
"step_number": 1,
|
||||
"total_steps": 2,
|
||||
"next_step_required": True,
|
||||
"findings": "Starting SSRF vulnerability analysis",
|
||||
"relevant_files": [self.api_file],
|
||||
"audit_focus": "owasp",
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if response1:
|
||||
self.logger.info(" ✅ Model selection recognized")
|
||||
|
||||
# Test 2: Skip expert analysis
|
||||
response2, _ = self.call_mcp_tool_direct(
|
||||
"secaudit",
|
||||
{
|
||||
"step": f"Complete security investigation of {self.auth_file}",
|
||||
"step_number": 1,
|
||||
"total_steps": 1,
|
||||
"next_step_required": False,
|
||||
"findings": "Security issues documented",
|
||||
"files_checked": [self.auth_file],
|
||||
"relevant_files": [self.auth_file],
|
||||
"confidence": "high",
|
||||
"use_assistant_model": False, # Skip expert analysis
|
||||
"model": "gemini-2.0-flash-lite",
|
||||
},
|
||||
)
|
||||
|
||||
if response2:
|
||||
try:
|
||||
response_data = json.loads(response2)
|
||||
# Should complete without expert analysis
|
||||
if response_data.get("status") == "security_analysis_complete":
|
||||
self.logger.info(" ✅ Skip expert analysis option works")
|
||||
return True
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
# Or might just complete the analysis
|
||||
response_lower = response2.lower()
|
||||
if "complete" in response_lower or "security" in response_lower:
|
||||
self.logger.info(" ✅ Analysis performed without expert model")
|
||||
return True
|
||||
|
||||
self.logger.error("Expected model selection or skip behavior")
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Model selection test failed: {e}")
|
||||
return False
|
||||
@@ -11,6 +11,7 @@ from .docgen_prompt import DOCGEN_PROMPT
|
||||
from .planner_prompt import PLANNER_PROMPT
|
||||
from .precommit_prompt import PRECOMMIT_PROMPT
|
||||
from .refactor_prompt import REFACTOR_PROMPT
|
||||
from .secaudit_prompt import SECAUDIT_PROMPT
|
||||
from .testgen_prompt import TESTGEN_PROMPT
|
||||
from .thinkdeep_prompt import THINKDEEP_PROMPT
|
||||
from .tracer_prompt import TRACER_PROMPT
|
||||
@@ -26,6 +27,7 @@ __all__ = [
|
||||
"PLANNER_PROMPT",
|
||||
"PRECOMMIT_PROMPT",
|
||||
"REFACTOR_PROMPT",
|
||||
"SECAUDIT_PROMPT",
|
||||
"TESTGEN_PROMPT",
|
||||
"TRACER_PROMPT",
|
||||
]
|
||||
|
||||
420
systemprompts/secaudit_prompt.py
Normal file
420
systemprompts/secaudit_prompt.py
Normal file
@@ -0,0 +1,420 @@
|
||||
"""
|
||||
SECAUDIT tool system prompt
|
||||
"""
|
||||
|
||||
SECAUDIT_PROMPT = """
|
||||
ROLE
|
||||
You are an expert security auditor receiving systematic investigation findings from Claude.
|
||||
Claude has performed methodical security analysis following comprehensive security audit methodology.
|
||||
Your role is to provide expert security analysis based on Claude's systematic investigation.
|
||||
|
||||
SYSTEMATIC SECURITY INVESTIGATION CONTEXT
|
||||
Claude has followed a systematic security audit approach:
|
||||
1. Security scope and attack surface analysis
|
||||
2. Authentication and authorization assessment
|
||||
3. Input validation and data handling security review
|
||||
4. OWASP Top 10 (2021) systematic evaluation
|
||||
5. Dependencies and infrastructure security analysis
|
||||
6. Compliance and risk assessment
|
||||
|
||||
You are receiving:
|
||||
1. Security audit scope and application context
|
||||
2. Claude's systematic security investigation findings
|
||||
3. Essential files identified as critical for security assessment
|
||||
4. Security issues discovered with severity classifications
|
||||
5. Compliance requirements and threat level assessment
|
||||
|
||||
CRITICAL LINE NUMBER INSTRUCTIONS
|
||||
Code is presented with line number markers "LINE│ code". These markers are for reference ONLY and MUST NOT be
|
||||
included in any code you generate. Always reference specific line numbers for Claude to locate
|
||||
exact positions if needed to point to exact locations. Include a very short code excerpt alongside for clarity.
|
||||
Include context_start_text and context_end_text as backup references. Never include "LINE│" markers in generated code
|
||||
snippets.
|
||||
|
||||
WORKFLOW CONTEXT
|
||||
Your task is to analyze Claude's systematic security investigation and provide expert security analysis back to Claude,
|
||||
who will then present the findings to the user in a consolidated format.
|
||||
|
||||
STRUCTURED JSON OUTPUT FORMAT
|
||||
You MUST respond with a properly formatted JSON object following this exact schema.
|
||||
Do NOT include any text before or after the JSON. The response must be valid JSON only.
|
||||
|
||||
IF MORE INFORMATION IS NEEDED:
|
||||
If you lack critical information to proceed, you MUST only respond with the following:
|
||||
{
|
||||
"status": "files_required_to_continue",
|
||||
"mandatory_instructions": "<your critical instructions for Claude>",
|
||||
"files_needed": ["[file name here]", "[or some folder/]"]
|
||||
}
|
||||
|
||||
FOR COMPLETE SECURITY ANALYSIS:
|
||||
{
|
||||
"status": "security_analysis_complete",
|
||||
"summary": "<brief description of the security posture and key findings>",
|
||||
"investigation_steps": [
|
||||
"<step 1: security scope and attack surface analysis>",
|
||||
"<step 2: authentication and authorization assessment>",
|
||||
"<step 3: input validation and data handling review>",
|
||||
"<step 4: OWASP Top 10 systematic evaluation>",
|
||||
"<step 5: dependencies and infrastructure analysis>",
|
||||
"<step 6: compliance and risk assessment>",
|
||||
"..."
|
||||
],
|
||||
"security_findings": [
|
||||
{
|
||||
"category": "<OWASP category or security domain>",
|
||||
"severity": "Critical|High|Medium|Low",
|
||||
"vulnerability": "<specific vulnerability name>",
|
||||
"description": "<technical description of the security issue>",
|
||||
"impact": "<potential business and technical impact>",
|
||||
"exploitability": "<how easily this can be exploited>",
|
||||
"evidence": "<code evidence or configuration showing the issue>",
|
||||
"remediation": "<specific steps to fix this vulnerability>",
|
||||
"timeline": "<recommended remediation timeline: immediate/short-term/medium-term>",
|
||||
"file_references": ["<file:line format for exact locations>"],
|
||||
"function_name": "<optional: specific function/method name if identified>",
|
||||
"start_line": "<optional: starting line number if specific location identified>",
|
||||
"end_line": "<optional: ending line number if specific location identified>",
|
||||
"context_start_text": "<optional: exact text from start line for verification>",
|
||||
"context_end_text": "<optional: exact text from end line for verification>"
|
||||
}
|
||||
],
|
||||
"owasp_assessment": {
|
||||
"A01_broken_access_control": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
},
|
||||
"A02_cryptographic_failures": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
},
|
||||
"A03_injection": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
},
|
||||
"A04_insecure_design": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
},
|
||||
"A05_security_misconfiguration": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
},
|
||||
"A06_vulnerable_components": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
},
|
||||
"A07_identification_authentication_failures": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
},
|
||||
"A08_software_data_integrity_failures": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
},
|
||||
"A09_security_logging_monitoring_failures": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
},
|
||||
"A10_server_side_request_forgery": {
|
||||
"status": "Vulnerable|Secure|Not_Applicable",
|
||||
"findings": ["<finding 1>", "<finding 2>"],
|
||||
"recommendations": ["<recommendation 1>", "<recommendation 2>"]
|
||||
}
|
||||
},
|
||||
"compliance_assessment": [
|
||||
{
|
||||
"framework": "<SOC2/PCI DSS/HIPAA/GDPR/etc>",
|
||||
"status": "Compliant|Non-Compliant|Partially Compliant|Not Applicable",
|
||||
"gaps": ["<specific compliance gap 1>", "<specific compliance gap 2>"],
|
||||
"recommendations": ["<compliance recommendation 1>", "<compliance recommendation 2>"]
|
||||
}
|
||||
],
|
||||
"risk_assessment": {
|
||||
"overall_risk_level": "Critical|High|Medium|Low",
|
||||
"threat_landscape": "<assessment of relevant threats for this application>",
|
||||
"attack_vectors": ["<primary attack vector 1>", "<primary attack vector 2>"],
|
||||
"business_impact": "<potential business consequences of identified vulnerabilities>",
|
||||
"likelihood_assessment": "<probability of successful attacks based on current security posture>"
|
||||
},
|
||||
"remediation_roadmap": [
|
||||
{
|
||||
"priority": "Critical|High|Medium|Low",
|
||||
"timeline": "Immediate|Short-term|Medium-term|Long-term",
|
||||
"effort": "Low|Medium|High",
|
||||
"description": "<remediation task description>",
|
||||
"dependencies": ["<dependency 1>", "<dependency 2>"],
|
||||
"success_criteria": "<how to validate this remediation>",
|
||||
"cost_impact": "<estimated cost and resource requirements>"
|
||||
}
|
||||
],
|
||||
"positive_security_findings": [
|
||||
"<security strength 1: well-implemented security controls>",
|
||||
"<security strength 2: good security practices observed>",
|
||||
"<security strength 3: proper security architecture decisions>"
|
||||
],
|
||||
"monitoring_recommendations": [
|
||||
"<monitoring recommendation 1: what to monitor for ongoing security>",
|
||||
"<monitoring recommendation 2: alerts and thresholds to implement>",
|
||||
"<monitoring recommendation 3: security metrics to track>"
|
||||
],
|
||||
"investigation_summary": "<comprehensive summary of the complete security audit process and final security posture assessment>"
|
||||
}
|
||||
|
||||
COMPREHENSIVE SECURITY ASSESSMENT METHODOLOGY
|
||||
|
||||
Your analysis must cover these critical security domains:
|
||||
|
||||
1. OWASP TOP 10 (2021) SYSTEMATIC EVALUATION:
|
||||
|
||||
A01 - BROKEN ACCESS CONTROL:
|
||||
• Authorization bypass vulnerabilities
|
||||
• Privilege escalation possibilities
|
||||
• Insecure direct object references
|
||||
• Missing function level access control
|
||||
• CORS misconfiguration
|
||||
• Force browsing to authenticated pages
|
||||
|
||||
A02 - CRYPTOGRAPHIC FAILURES:
|
||||
• Weak encryption algorithms or implementations
|
||||
• Hardcoded secrets and credentials
|
||||
• Insufficient protection of sensitive data
|
||||
• Weak key management practices
|
||||
• Plain text storage of sensitive information
|
||||
• Inadequate transport layer protection
|
||||
|
||||
A03 - INJECTION:
|
||||
• SQL injection vulnerabilities
|
||||
• Cross-site scripting (XSS) - stored, reflected, DOM-based
|
||||
• Command injection possibilities
|
||||
• LDAP injection vulnerabilities
|
||||
• NoSQL injection attacks
|
||||
• Header injection and response splitting
|
||||
|
||||
A04 - INSECURE DESIGN:
|
||||
• Missing threat modeling
|
||||
• Insecure design patterns
|
||||
• Business logic vulnerabilities
|
||||
• Missing security controls by design
|
||||
• Insufficient separation of concerns
|
||||
• Inadequate security requirements
|
||||
|
||||
A05 - SECURITY MISCONFIGURATION:
|
||||
• Default configurations not changed
|
||||
• Incomplete or ad hoc configurations
|
||||
• Open cloud storage permissions
|
||||
• Misconfigured HTTP headers
|
||||
• Verbose error messages containing sensitive information
|
||||
• Outdated or missing security patches
|
||||
|
||||
A06 - VULNERABLE AND OUTDATED COMPONENTS:
|
||||
• Components with known vulnerabilities
|
||||
• Outdated libraries and frameworks
|
||||
• Unsupported or end-of-life components
|
||||
• Unknown component inventory
|
||||
• Missing security patches
|
||||
• Insecure component configurations
|
||||
|
||||
A07 - IDENTIFICATION AND AUTHENTICATION FAILURES:
|
||||
• Weak password requirements
|
||||
• Session management vulnerabilities
|
||||
• Missing multi-factor authentication
|
||||
• Credential stuffing vulnerabilities
|
||||
• Session fixation attacks
|
||||
• Insecure password recovery mechanisms
|
||||
|
||||
A08 - SOFTWARE AND DATA INTEGRITY FAILURES:
|
||||
• Unsigned or unverified software updates
|
||||
• Insecure CI/CD pipelines
|
||||
• Auto-update functionality vulnerabilities
|
||||
• Untrusted deserialization
|
||||
• Missing integrity checks
|
||||
• Insufficient supply chain security
|
||||
|
||||
A09 - SECURITY LOGGING AND MONITORING FAILURES:
|
||||
• Insufficient logging of security events
|
||||
• Missing real-time monitoring
|
||||
• Inadequate incident response procedures
|
||||
• Log tampering possibilities
|
||||
• Missing audit trails
|
||||
• Delayed detection of security breaches
|
||||
|
||||
A10 - SERVER-SIDE REQUEST FORGERY (SSRF):
|
||||
• SSRF vulnerabilities in URL fetching
|
||||
• Missing input validation for URLs
|
||||
• Inadequate network segmentation
|
||||
• Blind SSRF scenarios
|
||||
• DNS rebinding attack possibilities
|
||||
• Cloud metadata service access
|
||||
|
||||
2. TECHNOLOGY-SPECIFIC SECURITY PATTERNS:
|
||||
|
||||
WEB APPLICATIONS:
|
||||
• Cross-Site Request Forgery (CSRF) protection
|
||||
• Cookie security attributes (HttpOnly, Secure, SameSite)
|
||||
• Content Security Policy (CSP) implementation
|
||||
• HTTP security headers (HSTS, X-Frame-Options, etc.)
|
||||
• Session management security
|
||||
• Input validation and output encoding
|
||||
• File upload security
|
||||
|
||||
API SECURITY:
|
||||
• Authentication and authorization mechanisms
|
||||
• Rate limiting and throttling
|
||||
• Input validation and sanitization
|
||||
• API versioning security considerations
|
||||
• Request/response validation
|
||||
• API key management and rotation
|
||||
• GraphQL security considerations
|
||||
|
||||
MOBILE APPLICATIONS:
|
||||
• Platform-specific security controls (iOS/Android)
|
||||
• Secure data storage practices
|
||||
• Certificate pinning implementation
|
||||
• Inter-app communication security
|
||||
• Runtime application self-protection
|
||||
• Binary protection and obfuscation
|
||||
• Mobile authentication patterns
|
||||
|
||||
CLOUD APPLICATIONS:
|
||||
• Identity and Access Management (IAM)
|
||||
• Container and orchestration security
|
||||
• Serverless security considerations
|
||||
• Infrastructure as Code security
|
||||
• Cloud storage and database security
|
||||
• Network security and segmentation
|
||||
• Secrets management in cloud environments
|
||||
|
||||
3. COMPLIANCE FRAMEWORK ASSESSMENT:
|
||||
|
||||
SOC2 TYPE II CONTROLS:
|
||||
• Access management and authorization controls
|
||||
• Data encryption and protection measures
|
||||
• System monitoring and incident response
|
||||
• Change management and deployment procedures
|
||||
• Vendor management and third-party security
|
||||
• Business continuity and disaster recovery
|
||||
|
||||
PCI DSS REQUIREMENTS:
|
||||
• Cardholder data protection and encryption
|
||||
• Secure payment processing workflows
|
||||
• Network security and segmentation
|
||||
• Regular security testing and vulnerability management
|
||||
• Strong access control measures
|
||||
• Comprehensive logging and monitoring
|
||||
|
||||
HIPAA SECURITY RULE:
|
||||
• Protected Health Information (PHI) safeguards
|
||||
• Access controls and user authentication
|
||||
• Audit controls and integrity protection
|
||||
• Transmission security for PHI
|
||||
• Assigned security responsibility
|
||||
• Information systems activity review
|
||||
|
||||
GDPR DATA PROTECTION:
|
||||
• Data protection by design and default
|
||||
• Lawful basis for data processing
|
||||
• Data subject rights implementation
|
||||
• Privacy impact assessments
|
||||
• Data breach notification procedures
|
||||
• Cross-border data transfer protections
|
||||
|
||||
4. RISK ASSESSMENT METHODOLOGY:
|
||||
|
||||
THREAT MODELING:
|
||||
• Asset identification and classification
|
||||
• Threat actor analysis and motivation
|
||||
• Attack vector enumeration and analysis
|
||||
• Impact assessment for identified threats
|
||||
• Likelihood evaluation based on current controls
|
||||
• Risk prioritization matrix (Impact × Likelihood)
|
||||
|
||||
VULNERABILITY PRIORITIZATION:
|
||||
• CVSS scoring for identified vulnerabilities
|
||||
• Business context and asset criticality
|
||||
• Exploit availability and complexity
|
||||
• Compensating controls effectiveness
|
||||
• Regulatory and compliance requirements
|
||||
• Cost-benefit analysis for remediation
|
||||
|
||||
5. REMEDIATION PLANNING:
|
||||
|
||||
IMMEDIATE ACTIONS (0-30 days):
|
||||
• Critical vulnerability patches
|
||||
• Emergency configuration changes
|
||||
• Incident response activation
|
||||
• Temporary compensating controls
|
||||
|
||||
SHORT-TERM FIXES (1-3 months):
|
||||
• Security control implementations
|
||||
• Process improvements
|
||||
• Training and awareness programs
|
||||
• Monitoring and alerting enhancements
|
||||
|
||||
MEDIUM-TERM IMPROVEMENTS (3-12 months):
|
||||
• Architecture and design changes
|
||||
• Technology upgrades and migrations
|
||||
• Compliance program maturation
|
||||
• Security culture development
|
||||
|
||||
LONG-TERM STRATEGIC INITIATIVES (1+ years):
|
||||
• Security transformation programs
|
||||
• Zero-trust architecture implementation
|
||||
• Advanced threat protection capabilities
|
||||
• Continuous security improvement processes
|
||||
|
||||
CRITICAL SECURITY AUDIT PRINCIPLES:
|
||||
1. Security vulnerabilities can ONLY be identified from actual code and configuration - never fabricated or assumed
|
||||
2. Focus ONLY on security-related issues - avoid suggesting general code improvements unrelated to security
|
||||
3. Propose specific, actionable security fixes that address identified vulnerabilities without introducing new risks
|
||||
4. Document security analysis systematically for audit trail and compliance purposes
|
||||
5. Rank security findings by risk (likelihood × impact) based on evidence from actual code and configuration
|
||||
6. Always include specific file:line references for exact vulnerability locations when available
|
||||
7. Consider the application context when assessing risk (internal tool vs public-facing vs regulated industry)
|
||||
8. Provide both technical remediation steps and business impact assessment for each finding
|
||||
9. Focus on practical, implementable security improvements rather than theoretical best practices
|
||||
10. Ensure remediation recommendations are proportionate to the actual risk and business requirements
|
||||
|
||||
PRECISION SECURITY REFERENCES:
|
||||
When you identify specific vulnerability locations, include optional precision fields:
|
||||
- function_name: The exact function/method name where the vulnerability exists
|
||||
- start_line/end_line: Line numbers from the LINE│ markers (for reference ONLY - never include LINE│ in generated code)
|
||||
- context_start_text/context_end_text: Exact text from those lines for verification
|
||||
- These fields help Claude locate exact positions for implementing security fixes
|
||||
|
||||
REMEDIATION SAFETY AND VALIDATION:
|
||||
Before suggesting any security fix, thoroughly analyze the proposed change to ensure it does not:
|
||||
- Introduce new vulnerabilities or security weaknesses
|
||||
- Break existing functionality or user workflows
|
||||
- Create performance or availability issues
|
||||
- Conflict with business requirements or compliance needs
|
||||
- Bypass necessary business logic or validation steps
|
||||
- Impact related security controls or dependencies
|
||||
|
||||
Consider for each remediation:
|
||||
- Root cause analysis to address underlying issues
|
||||
- Defense in depth and layered security approaches
|
||||
- Backward compatibility and migration strategies
|
||||
- Testing and validation procedures
|
||||
- Rollback plans for failed implementations
|
||||
- Documentation and knowledge transfer requirements
|
||||
|
||||
Your security analysis should generate comprehensive, risk-prioritized findings with emphasis on:
|
||||
- Identifying exact vulnerabilities with concrete evidence
|
||||
- Implementing targeted, safe remediation strategies
|
||||
- Maintaining detailed audit trails and documentation
|
||||
- Providing actionable business impact assessments
|
||||
- Ensuring compliance with relevant security standards
|
||||
- Establishing ongoing security monitoring and improvement processes
|
||||
|
||||
Remember: A thorough security audit not only identifies current vulnerabilities but also establishes a foundation for continuous security improvement and risk management.
|
||||
"""
|
||||
70
test_simulation_files/api_endpoints.py
Normal file
70
test_simulation_files/api_endpoints.py
Normal file
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env python3
|
||||
from flask import Flask, request, jsonify
|
||||
import os
|
||||
import subprocess
|
||||
import requests
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
# A05: Security Misconfiguration - Debug mode enabled
|
||||
app.config['DEBUG'] = True
|
||||
app.config['SECRET_KEY'] = 'dev-secret-key' # Hardcoded secret
|
||||
|
||||
@app.route('/api/search', methods=['GET'])
|
||||
def search():
|
||||
'''Search endpoint with multiple vulnerabilities'''
|
||||
# A03: Injection - XSS vulnerability, no input sanitization
|
||||
query = request.args.get('q', '')
|
||||
|
||||
# A03: Injection - Command injection vulnerability
|
||||
if 'file:' in query:
|
||||
filename = query.split('file:')[1]
|
||||
# Direct command execution
|
||||
result = subprocess.run(f"cat {filename}", shell=True, capture_output=True, text=True)
|
||||
return jsonify({"result": result.stdout})
|
||||
|
||||
# A10: Server-Side Request Forgery (SSRF)
|
||||
if query.startswith('http'):
|
||||
# No validation of URL, allows internal network access
|
||||
response = requests.get(query)
|
||||
return jsonify({"content": response.text})
|
||||
|
||||
# Return search results without output encoding
|
||||
return f"<h1>Search Results for: {query}</h1>"
|
||||
|
||||
@app.route('/api/admin', methods=['GET'])
|
||||
def admin_panel():
|
||||
'''Admin panel with broken access control'''
|
||||
# A01: Broken Access Control - No authentication check
|
||||
# Anyone can access admin functionality
|
||||
action = request.args.get('action')
|
||||
|
||||
if action == 'delete_user':
|
||||
user_id = request.args.get('user_id')
|
||||
# Performs privileged action without authorization
|
||||
return jsonify({"status": "User deleted", "user_id": user_id})
|
||||
|
||||
return jsonify({"status": "Admin panel"})
|
||||
|
||||
@app.route('/api/upload', methods=['POST'])
|
||||
def upload_file():
|
||||
'''File upload with security issues'''
|
||||
# A05: Security Misconfiguration - No file type validation
|
||||
file = request.files.get('file')
|
||||
if file:
|
||||
# Saves any file type to server
|
||||
filename = file.filename
|
||||
file.save(os.path.join('/tmp', filename))
|
||||
|
||||
# A03: Path traversal vulnerability
|
||||
return jsonify({"status": "File uploaded", "path": f"/tmp/{filename}"})
|
||||
|
||||
return jsonify({"error": "No file provided"})
|
||||
|
||||
# A06: Vulnerable and Outdated Components
|
||||
# Using old Flask version with known vulnerabilities (hypothetical)
|
||||
# requirements.txt: Flask==0.12.2 (known security issues)
|
||||
|
||||
if __name__ == '__main__':
|
||||
# A05: Security Misconfiguration - Running on all interfaces
|
||||
app.run(host='0.0.0.0', port=5000, debug=True)
|
||||
60
test_simulation_files/auth_manager.py
Normal file
60
test_simulation_files/auth_manager.py
Normal file
@@ -0,0 +1,60 @@
|
||||
#!/usr/bin/env python3
|
||||
import hashlib
|
||||
import pickle
|
||||
import sqlite3
|
||||
from flask import request, session
|
||||
|
||||
class AuthenticationManager:
|
||||
def __init__(self, db_path="users.db"):
|
||||
# A01: Broken Access Control - No proper session management
|
||||
self.db_path = db_path
|
||||
self.sessions = {} # In-memory session storage
|
||||
def login(self, username, password):
|
||||
'''User login with various security vulnerabilities'''
|
||||
# A03: Injection - SQL injection vulnerability
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Direct string interpolation in SQL query
|
||||
query = f"SELECT id, password_hash FROM users WHERE username = '{username}'"
|
||||
cursor.execute(query)
|
||||
|
||||
user = cursor.fetchone()
|
||||
if not user:
|
||||
return {"status": "failed", "message": "User not found"}
|
||||
|
||||
# A02: Cryptographic Failures - Weak hashing algorithm
|
||||
password_hash = hashlib.md5(password.encode()).hexdigest()
|
||||
|
||||
if user[1] == password_hash:
|
||||
# A07: Identification and Authentication Failures - Weak session generation
|
||||
session_id = hashlib.md5(f"{username}{password}".encode()).hexdigest()
|
||||
self.sessions[session_id] = {"user_id": user[0], "username": username}
|
||||
|
||||
return {"status": "success", "session_id": session_id}
|
||||
else:
|
||||
return {"status": "failed", "message": "Invalid password"}
|
||||
|
||||
def reset_password(self, email):
|
||||
'''Password reset with security issues'''
|
||||
# A04: Insecure Design - No rate limiting or validation
|
||||
reset_token = hashlib.md5(email.encode()).hexdigest()
|
||||
|
||||
# A09: Security Logging and Monitoring Failures - No security event logging
|
||||
# Simply returns token without any verification or logging
|
||||
return {"reset_token": reset_token, "url": f"/reset?token={reset_token}"}
|
||||
|
||||
def deserialize_user_data(self, data):
|
||||
'''Unsafe deserialization'''
|
||||
# A08: Software and Data Integrity Failures - Insecure deserialization
|
||||
return pickle.loads(data)
|
||||
|
||||
def get_user_profile(self, user_id):
|
||||
'''Get user profile with authorization issues'''
|
||||
# A01: Broken Access Control - No authorization check
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Fetches any user profile without checking permissions
|
||||
cursor.execute("SELECT * FROM users WHERE id = ?", (user_id,))
|
||||
return cursor.fetchone()
|
||||
16
test_simulation_files/config.json
Normal file
16
test_simulation_files/config.json
Normal file
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"database": {
|
||||
"host": "localhost",
|
||||
"port": 5432,
|
||||
"name": "testdb",
|
||||
"ssl": true
|
||||
},
|
||||
"cache": {
|
||||
"redis_url": "redis://localhost:6379",
|
||||
"ttl": 3600
|
||||
},
|
||||
"logging": {
|
||||
"level": "INFO",
|
||||
"format": "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
|
||||
}
|
||||
}
|
||||
32
test_simulation_files/test_module.py
Normal file
32
test_simulation_files/test_module.py
Normal file
@@ -0,0 +1,32 @@
|
||||
"""
|
||||
Sample Python module for testing MCP conversation continuity
|
||||
"""
|
||||
|
||||
def fibonacci(n):
|
||||
"""Calculate fibonacci number recursively"""
|
||||
if n <= 1:
|
||||
return n
|
||||
return fibonacci(n-1) + fibonacci(n-2)
|
||||
|
||||
def factorial(n):
|
||||
"""Calculate factorial iteratively"""
|
||||
result = 1
|
||||
for i in range(1, n + 1):
|
||||
result *= i
|
||||
return result
|
||||
|
||||
class Calculator:
|
||||
"""Simple calculator class"""
|
||||
|
||||
def __init__(self):
|
||||
self.history = []
|
||||
|
||||
def add(self, a, b):
|
||||
result = a + b
|
||||
self.history.append(f"{a} + {b} = {result}")
|
||||
return result
|
||||
|
||||
def multiply(self, a, b):
|
||||
result = a * b
|
||||
self.history.append(f"{a} * {b} = {result}")
|
||||
return result
|
||||
452
tests/test_secaudit.py
Normal file
452
tests/test_secaudit.py
Normal file
@@ -0,0 +1,452 @@
|
||||
"""
|
||||
Tests for the secaudit tool using WorkflowTool architecture.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from tools.models import ToolModelCategory
|
||||
from tools.secaudit import SecauditRequest, SecauditTool
|
||||
|
||||
|
||||
class TestSecauditTool:
|
||||
"""Test suite for SecauditTool using WorkflowTool architecture."""
|
||||
|
||||
def test_tool_metadata(self):
|
||||
"""Test basic tool metadata and configuration."""
|
||||
tool = SecauditTool()
|
||||
|
||||
assert tool.get_name() == "secaudit"
|
||||
assert "COMPREHENSIVE SECURITY AUDIT" in tool.get_description()
|
||||
assert tool.get_default_temperature() == 0.2 # TEMPERATURE_ANALYTICAL
|
||||
assert tool.get_model_category() == ToolModelCategory.EXTENDED_REASONING
|
||||
assert tool.requires_model() is True
|
||||
|
||||
def test_request_validation(self):
|
||||
"""Test Pydantic request model validation."""
|
||||
# Valid security audit step request
|
||||
step_request = SecauditRequest(
|
||||
step="Beginning comprehensive security audit of web application",
|
||||
step_number=1,
|
||||
total_steps=6,
|
||||
next_step_required=True,
|
||||
findings="Identified React/Node.js e-commerce application with payment processing",
|
||||
files_checked=["/src/auth.py", "/src/payment.py"],
|
||||
relevant_files=["/src/auth.py", "/src/payment.py"],
|
||||
relevant_context=["AuthController.login", "PaymentService.process"],
|
||||
security_scope="Web application - e-commerce platform",
|
||||
threat_level="high",
|
||||
compliance_requirements=["PCI DSS", "SOC2"],
|
||||
audit_focus="comprehensive",
|
||||
confidence="medium",
|
||||
)
|
||||
|
||||
assert step_request.step_number == 1
|
||||
assert step_request.threat_level == "high"
|
||||
assert step_request.compliance_requirements == ["PCI DSS", "SOC2"]
|
||||
assert step_request.audit_focus == "comprehensive"
|
||||
assert len(step_request.relevant_context) == 2
|
||||
|
||||
def test_request_validation_defaults(self):
|
||||
"""Test default values for optional fields."""
|
||||
minimal_request = SecauditRequest(
|
||||
step="Security audit step",
|
||||
step_number=1,
|
||||
total_steps=4,
|
||||
next_step_required=True,
|
||||
findings="Initial findings",
|
||||
)
|
||||
|
||||
assert minimal_request.threat_level == "medium" # Default value
|
||||
assert minimal_request.audit_focus == "comprehensive" # Default value
|
||||
assert minimal_request.confidence == "low" # Default value
|
||||
assert minimal_request.compliance_requirements == [] # Default empty list
|
||||
|
||||
def test_request_validation_invalid_threat_level(self):
|
||||
"""Test validation with invalid threat level."""
|
||||
with pytest.raises(ValueError):
|
||||
SecauditRequest(
|
||||
step="Security audit step",
|
||||
step_number=1,
|
||||
total_steps=4,
|
||||
next_step_required=True,
|
||||
findings="Initial findings",
|
||||
threat_level="invalid", # Should only accept low, medium, high, critical
|
||||
)
|
||||
|
||||
def test_request_validation_invalid_audit_focus(self):
|
||||
"""Test validation with invalid audit focus."""
|
||||
with pytest.raises(ValueError):
|
||||
SecauditRequest(
|
||||
step="Security audit step",
|
||||
step_number=1,
|
||||
total_steps=4,
|
||||
next_step_required=True,
|
||||
findings="Initial findings",
|
||||
audit_focus="invalid", # Should only accept defined options
|
||||
)
|
||||
|
||||
def test_input_schema_generation(self):
|
||||
"""Test that input schema is generated correctly."""
|
||||
tool = SecauditTool()
|
||||
schema = tool.get_input_schema()
|
||||
|
||||
# Verify required security audit fields are present
|
||||
assert "step" in schema["properties"]
|
||||
assert "step_number" in schema["properties"]
|
||||
assert "total_steps" in schema["properties"]
|
||||
assert "next_step_required" in schema["properties"]
|
||||
assert "findings" in schema["properties"]
|
||||
|
||||
# Verify security-specific fields
|
||||
assert "security_scope" in schema["properties"]
|
||||
assert "threat_level" in schema["properties"]
|
||||
assert "compliance_requirements" in schema["properties"]
|
||||
assert "audit_focus" in schema["properties"]
|
||||
|
||||
# Verify field types
|
||||
assert schema["properties"]["threat_level"]["type"] == "string"
|
||||
assert schema["properties"]["compliance_requirements"]["type"] == "array"
|
||||
|
||||
def test_step_guidance_step_1(self):
|
||||
"""Test step-specific guidance for step 1 (Security Scope Analysis)."""
|
||||
tool = SecauditTool()
|
||||
request = SecauditRequest(
|
||||
step="Begin security audit",
|
||||
step_number=1,
|
||||
total_steps=6,
|
||||
next_step_required=True,
|
||||
findings="Starting security assessment",
|
||||
)
|
||||
|
||||
actions = tool.get_required_actions(
|
||||
request.step_number, request.confidence, request.findings, request.total_steps
|
||||
)
|
||||
|
||||
assert len(actions) == 4
|
||||
assert "Identify application type, technology stack, and security scope" in actions
|
||||
assert "Map attack surface, entry points, and data flows" in actions
|
||||
assert "Determine relevant security standards and compliance requirements" in actions
|
||||
assert "Establish threat landscape and risk context for the application" in actions
|
||||
|
||||
def test_step_guidance_step_2(self):
|
||||
"""Test step-specific guidance for step 2 (Authentication Assessment)."""
|
||||
tool = SecauditTool()
|
||||
request = SecauditRequest(
|
||||
step="Analyze authentication",
|
||||
step_number=2,
|
||||
total_steps=6,
|
||||
next_step_required=True,
|
||||
findings="Authentication analysis",
|
||||
)
|
||||
|
||||
actions = tool.get_required_actions(
|
||||
request.step_number, request.confidence, request.findings, request.total_steps
|
||||
)
|
||||
|
||||
assert len(actions) == 4
|
||||
assert "Analyze authentication mechanisms and session management" in actions
|
||||
assert "Check authorization controls, access patterns, and privilege escalation risks" in actions
|
||||
assert "Assess multi-factor authentication, password policies, and account security" in actions
|
||||
assert "Review identity and access management implementations" in actions
|
||||
|
||||
def test_step_guidance_step_4(self):
|
||||
"""Test step-specific guidance for step 4 (OWASP Top 10 Review)."""
|
||||
tool = SecauditTool()
|
||||
request = SecauditRequest(
|
||||
step="OWASP Top 10 review", step_number=4, total_steps=6, next_step_required=True, findings="OWASP analysis"
|
||||
)
|
||||
|
||||
actions = tool.get_required_actions(
|
||||
request.step_number, request.confidence, request.findings, request.total_steps
|
||||
)
|
||||
|
||||
assert len(actions) == 4
|
||||
assert "Conduct OWASP Top 10 (2021) systematic review across all categories" in actions
|
||||
assert "Check each OWASP category methodically with specific findings and evidence" in actions
|
||||
assert "Cross-reference findings with application context and technology stack" in actions
|
||||
assert "Prioritize vulnerabilities based on exploitability and business impact" in actions
|
||||
|
||||
def test_expert_analysis_trigger(self):
|
||||
"""Test when expert analysis should be triggered."""
|
||||
tool = SecauditTool()
|
||||
|
||||
# Create a mock consolidated findings object
|
||||
class MockConsolidatedFindings:
|
||||
def __init__(self, relevant_files=None, findings=None, issues_found=None):
|
||||
self.relevant_files = relevant_files or []
|
||||
self.findings = findings or []
|
||||
self.issues_found = issues_found or []
|
||||
|
||||
# Should trigger expert analysis when we have meaningful findings
|
||||
findings_with_files = MockConsolidatedFindings(
|
||||
relevant_files=["/src/auth.py", "/src/payment.py"],
|
||||
findings=["Finding 1", "Finding 2"],
|
||||
issues_found=[{"severity": "high", "description": "SQL injection"}],
|
||||
)
|
||||
assert tool.should_call_expert_analysis(findings_with_files) is True
|
||||
|
||||
# Should trigger with just findings
|
||||
findings_only = MockConsolidatedFindings(findings=["Finding 1", "Finding 2"])
|
||||
assert tool.should_call_expert_analysis(findings_only) is True
|
||||
|
||||
# Should trigger with just issues
|
||||
issues_only = MockConsolidatedFindings(issues_found=[{"severity": "high", "description": "SQL injection"}])
|
||||
assert tool.should_call_expert_analysis(issues_only) is True
|
||||
|
||||
# Should not trigger with no meaningful data
|
||||
no_findings = MockConsolidatedFindings()
|
||||
assert tool.should_call_expert_analysis(no_findings) is False
|
||||
|
||||
def test_expert_analysis_context_preparation(self):
|
||||
"""Test expert analysis context preparation."""
|
||||
tool = SecauditTool()
|
||||
|
||||
# Create a mock consolidated findings object
|
||||
class MockConsolidatedFindings:
|
||||
def __init__(self):
|
||||
self.hypotheses = []
|
||||
self.files_checked = ["/app/auth.py", "/app/payment.py", "/app/api.py", "/app/db.py"]
|
||||
self.relevant_files = ["/app/auth.py", "/app/payment.py", "/app/api.py"]
|
||||
self.relevant_context = ["AuthController.login", "PaymentService.process", "APIController.validate"]
|
||||
self.issues_found = [
|
||||
{"severity": "critical", "description": "SQL injection vulnerability in login endpoint"},
|
||||
{"severity": "high", "description": "Missing input validation in payment processing"},
|
||||
{"severity": "medium", "description": "Weak session management configuration"},
|
||||
]
|
||||
self.findings = [
|
||||
"Step 1: Identified e-commerce web application with payment processing",
|
||||
"Step 2: Found authentication vulnerabilities",
|
||||
"Step 3: Discovered input validation issues",
|
||||
]
|
||||
self.hypotheses = [
|
||||
{"step": 1, "confidence": "low", "hypothesis": "Initial security assessment"},
|
||||
{"step": 2, "confidence": "medium", "hypothesis": "Authentication issues confirmed"},
|
||||
{"step": 3, "confidence": "high", "hypothesis": "Multiple security vulnerabilities identified"},
|
||||
]
|
||||
self.images = []
|
||||
|
||||
# Set initial request to provide context
|
||||
tool.initial_request = "Perform security audit of e-commerce web application"
|
||||
tool.security_config = {
|
||||
"security_scope": "Web application - e-commerce platform with payment processing",
|
||||
"threat_level": "high",
|
||||
"compliance_requirements": ["PCI DSS", "SOC2", "GDPR"],
|
||||
"audit_focus": "comprehensive",
|
||||
"severity_filter": "all",
|
||||
}
|
||||
|
||||
consolidated_findings = MockConsolidatedFindings()
|
||||
context = tool.prepare_expert_analysis_context(consolidated_findings)
|
||||
|
||||
# Verify context contains all security-specific information
|
||||
assert "SECURITY AUDIT REQUEST" in context
|
||||
assert "Perform security audit of e-commerce web application" in context
|
||||
assert "SECURITY CONFIGURATION" in context
|
||||
assert "security_scope: Web application - e-commerce platform with payment processing" in context
|
||||
assert "threat_level: high" in context
|
||||
assert "compliance_requirements: ['PCI DSS', 'SOC2', 'GDPR']" in context
|
||||
assert "/app/auth.py" in context
|
||||
assert "AuthController.login" in context
|
||||
assert "CRITICAL SEVERITY:" in context
|
||||
assert "SQL injection vulnerability" in context
|
||||
assert "HIGH SEVERITY:" in context
|
||||
assert "Missing input validation" in context
|
||||
|
||||
def test_security_issues_formatting_empty(self):
|
||||
"""Test security issues formatting with no issues."""
|
||||
tool = SecauditTool()
|
||||
formatted = tool._format_security_issues([])
|
||||
assert "No security issues identified during systematic investigation." in formatted
|
||||
|
||||
def test_security_issues_formatting_with_issues(self):
|
||||
"""Test security issues formatting with multiple severity levels."""
|
||||
tool = SecauditTool()
|
||||
issues = [
|
||||
{"severity": "critical", "description": "Remote code execution vulnerability"},
|
||||
{"severity": "high", "description": "Authentication bypass"},
|
||||
{"severity": "medium", "description": "Information disclosure"},
|
||||
{"severity": "low", "description": "Missing security headers"},
|
||||
{"severity": "unknown", "description": "Unclassified issue"}, # Should go to low
|
||||
]
|
||||
|
||||
formatted = tool._format_security_issues(issues)
|
||||
|
||||
assert "CRITICAL SEVERITY:" in formatted
|
||||
assert "Remote code execution vulnerability" in formatted
|
||||
assert "HIGH SEVERITY:" in formatted
|
||||
assert "Authentication bypass" in formatted
|
||||
assert "MEDIUM SEVERITY:" in formatted
|
||||
assert "Information disclosure" in formatted
|
||||
assert "LOW SEVERITY:" in formatted
|
||||
assert "Missing security headers" in formatted
|
||||
assert "[UNKNOWN] Unclassified issue" in formatted
|
||||
|
||||
def test_tool_field_definitions(self):
|
||||
"""Test that all security-specific tool fields are properly defined."""
|
||||
tool = SecauditTool()
|
||||
fields = tool.get_tool_fields()
|
||||
|
||||
# Verify all expected fields are present
|
||||
expected_fields = [
|
||||
"step",
|
||||
"step_number",
|
||||
"total_steps",
|
||||
"next_step_required",
|
||||
"findings",
|
||||
"files_checked",
|
||||
"relevant_files",
|
||||
"relevant_context",
|
||||
"issues_found",
|
||||
"confidence",
|
||||
"backtrack_from_step",
|
||||
"images",
|
||||
"security_scope",
|
||||
"threat_level",
|
||||
"compliance_requirements",
|
||||
"audit_focus",
|
||||
"severity_filter",
|
||||
]
|
||||
|
||||
for field in expected_fields:
|
||||
assert field in fields, f"Field '{field}' not found in tool field definitions"
|
||||
|
||||
# Verify field descriptions are comprehensive
|
||||
assert "OWASP Top 10" in fields["step"]
|
||||
assert "security implications" in fields["step"]
|
||||
assert "threat vectors" in fields["step"]
|
||||
assert "application context" in fields["security_scope"]
|
||||
assert "threat level" in fields["threat_level"]
|
||||
assert "compliance frameworks" in fields["compliance_requirements"]
|
||||
|
||||
def test_workflow_request_model(self):
|
||||
"""Test that the workflow request model is correctly configured."""
|
||||
tool = SecauditTool()
|
||||
request_model = tool.get_workflow_request_model()
|
||||
assert request_model == SecauditRequest
|
||||
|
||||
def test_workflow_system_prompt(self):
|
||||
"""Test that the workflow system prompt is correctly configured."""
|
||||
tool = SecauditTool()
|
||||
system_prompt = tool.get_system_prompt()
|
||||
|
||||
# Verify it contains key security audit elements
|
||||
assert "OWASP Top 10" in system_prompt
|
||||
assert "security_analysis_complete" in system_prompt
|
||||
assert "vulnerability" in system_prompt
|
||||
assert "compliance_assessment" in system_prompt
|
||||
|
||||
def test_compliance_requirements_validation(self):
|
||||
"""Test compliance requirements validation in model validator."""
|
||||
# Test with valid compliance requirements
|
||||
valid_request = SecauditRequest(
|
||||
step="Security audit with compliance",
|
||||
step_number=1,
|
||||
total_steps=6,
|
||||
next_step_required=True,
|
||||
findings="Starting audit",
|
||||
compliance_requirements=["SOC2", "PCI DSS", "HIPAA"],
|
||||
)
|
||||
assert valid_request.compliance_requirements == ["SOC2", "PCI DSS", "HIPAA"]
|
||||
|
||||
# Test with unknown compliance requirement (should warn but not fail)
|
||||
unknown_compliance_request = SecauditRequest(
|
||||
step="Security audit with unknown compliance",
|
||||
step_number=1,
|
||||
total_steps=6,
|
||||
next_step_required=True,
|
||||
findings="Starting audit",
|
||||
compliance_requirements=["UNKNOWN_COMPLIANCE"],
|
||||
)
|
||||
# Should still create the request but log a warning
|
||||
assert unknown_compliance_request.compliance_requirements == ["UNKNOWN_COMPLIANCE"]
|
||||
|
||||
def test_comprehensive_workflow_scenario(self):
|
||||
"""Test a complete workflow scenario from start to finish."""
|
||||
tool = SecauditTool()
|
||||
|
||||
# Step 1: Initial security scope analysis
|
||||
step1_request = SecauditRequest(
|
||||
step="Begin comprehensive security audit of e-commerce web application",
|
||||
step_number=1,
|
||||
total_steps=6,
|
||||
next_step_required=True,
|
||||
findings="Identified Node.js/React application with payment processing and user management",
|
||||
security_scope="Web application - e-commerce platform",
|
||||
threat_level="high",
|
||||
compliance_requirements=["PCI DSS"],
|
||||
relevant_files=["/src/auth.js", "/src/payment.js"],
|
||||
)
|
||||
|
||||
step1_actions = tool.get_required_actions(
|
||||
step1_request.step_number, step1_request.confidence, step1_request.findings, step1_request.total_steps
|
||||
)
|
||||
assert "Identify application type" in step1_actions[0]
|
||||
|
||||
# Test should_call_expert_analysis with mock consolidated findings
|
||||
class MockConsolidatedFindings:
|
||||
def __init__(self):
|
||||
self.hypotheses = []
|
||||
self.relevant_files = []
|
||||
self.findings = []
|
||||
self.issues_found = []
|
||||
|
||||
mock_findings = MockConsolidatedFindings()
|
||||
assert not tool.should_call_expert_analysis(mock_findings)
|
||||
|
||||
# Step 6: Final assessment
|
||||
step6_request = SecauditRequest(
|
||||
step="Complete security assessment and risk evaluation",
|
||||
step_number=6,
|
||||
total_steps=6,
|
||||
next_step_required=False,
|
||||
findings="Comprehensive security audit completed with findings documented",
|
||||
security_scope="Web application - e-commerce platform",
|
||||
threat_level="high",
|
||||
compliance_requirements=["PCI DSS"],
|
||||
relevant_files=["/src/auth.js", "/src/payment.js", "/src/api.js"],
|
||||
relevant_context=["AuthService.authenticate", "PaymentProcessor.charge"],
|
||||
issues_found=[
|
||||
{"severity": "high", "description": "SQL injection in user search"},
|
||||
{"severity": "medium", "description": "Weak password policy"},
|
||||
],
|
||||
confidence="high",
|
||||
)
|
||||
|
||||
step6_actions = tool.get_required_actions(
|
||||
step6_request.step_number, step6_request.confidence, step6_request.findings, step6_request.total_steps
|
||||
)
|
||||
assert "Evaluate compliance requirements" in step6_actions[0]
|
||||
|
||||
# Create mock consolidated findings for final step
|
||||
final_findings = MockConsolidatedFindings()
|
||||
final_findings.relevant_files = step6_request.relevant_files
|
||||
final_findings.findings = ["Comprehensive security audit completed with findings documented"]
|
||||
final_findings.issues_found = step6_request.issues_found
|
||||
final_findings.relevant_context = []
|
||||
final_findings.images = []
|
||||
assert tool.should_call_expert_analysis(final_findings)
|
||||
|
||||
# Test expert analysis context generation with mock consolidated findings
|
||||
# Set up tool state as it would be after processing
|
||||
tool.initial_request = "Complete security assessment and risk evaluation"
|
||||
tool.security_config = {
|
||||
"security_scope": step6_request.security_scope,
|
||||
"threat_level": step6_request.threat_level,
|
||||
"compliance_requirements": step6_request.compliance_requirements,
|
||||
"audit_focus": step6_request.audit_focus,
|
||||
"severity_filter": step6_request.severity_filter,
|
||||
}
|
||||
|
||||
# Create a complete mock consolidated findings
|
||||
complete_findings = MockConsolidatedFindings()
|
||||
complete_findings.relevant_files = step6_request.relevant_files
|
||||
complete_findings.relevant_context = step6_request.relevant_context
|
||||
complete_findings.issues_found = step6_request.issues_found
|
||||
complete_findings.findings = ["Security audit findings from all steps"]
|
||||
complete_findings.files_checked = []
|
||||
complete_findings.images = []
|
||||
|
||||
context = tool.prepare_expert_analysis_context(complete_findings)
|
||||
assert "PCI DSS" in context
|
||||
assert "SQL injection" in context
|
||||
assert "HIGH SEVERITY:" in context
|
||||
@@ -12,6 +12,7 @@ from .listmodels import ListModelsTool
|
||||
from .planner import PlannerTool
|
||||
from .precommit import PrecommitTool
|
||||
from .refactor import RefactorTool
|
||||
from .secaudit import SecauditTool
|
||||
from .testgen import TestGenTool
|
||||
from .thinkdeep import ThinkDeepTool
|
||||
from .tracer import TracerTool
|
||||
@@ -29,6 +30,7 @@ __all__ = [
|
||||
"PlannerTool",
|
||||
"PrecommitTool",
|
||||
"RefactorTool",
|
||||
"SecauditTool",
|
||||
"TestGenTool",
|
||||
"TracerTool",
|
||||
"VersionTool",
|
||||
|
||||
781
tools/secaudit.py
Normal file
781
tools/secaudit.py
Normal file
@@ -0,0 +1,781 @@
|
||||
"""
|
||||
SECAUDIT Workflow tool - Comprehensive security audit with systematic investigation
|
||||
|
||||
This tool provides a structured workflow for comprehensive security assessment and analysis.
|
||||
It guides Claude through systematic investigation steps with forced pauses between each step
|
||||
to ensure thorough security examination, vulnerability identification, and compliance assessment
|
||||
before proceeding. The tool supports complex security scenarios including OWASP Top 10 coverage,
|
||||
compliance framework mapping, and technology-specific security patterns.
|
||||
|
||||
Key features:
|
||||
- Step-by-step security audit workflow with progress tracking
|
||||
- Context-aware file embedding (references during investigation, full content for analysis)
|
||||
- Automatic security issue tracking with severity classification
|
||||
- Expert analysis integration with external models
|
||||
- Support for focused security audits (OWASP, compliance, technology-specific)
|
||||
- Confidence-based workflow optimization
|
||||
- Risk-based prioritization and remediation planning
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, Any, Literal, Optional
|
||||
|
||||
from pydantic import Field, model_validator
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from tools.models import ToolModelCategory
|
||||
|
||||
from config import TEMPERATURE_ANALYTICAL
|
||||
from systemprompts import SECAUDIT_PROMPT
|
||||
from tools.shared.base_models import WorkflowRequest
|
||||
|
||||
from .workflow.base import WorkflowTool
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Tool-specific field descriptions for security audit workflow
|
||||
SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS = {
|
||||
"step": (
|
||||
"Describe what you're currently investigating for security audit by thinking deeply about security "
|
||||
"implications, threat vectors, and protection mechanisms. In step 1, clearly state your security "
|
||||
"audit plan and begin forming a systematic approach after identifying the application type, "
|
||||
"technology stack, and relevant security requirements. You must begin by passing the file path "
|
||||
"for the initial code you are about to audit in relevant_files. CRITICAL: Follow the OWASP Top 10 "
|
||||
"systematic checklist, examine authentication/authorization mechanisms, analyze input validation "
|
||||
"and data handling, assess dependency vulnerabilities, and evaluate infrastructure security. "
|
||||
"Consider not only obvious vulnerabilities but also subtle security gaps, configuration issues, "
|
||||
"design flaws, and compliance requirements. Map out the attack surface, understand the threat "
|
||||
"landscape, and identify areas requiring deeper security analysis. In all later steps, continue "
|
||||
"exploring with precision: trace security dependencies, verify security assumptions, and adapt "
|
||||
"your understanding as you uncover security evidence."
|
||||
),
|
||||
"step_number": (
|
||||
"The index of the current step in the security audit sequence, beginning at 1. Each step should "
|
||||
"build upon or revise the previous one."
|
||||
),
|
||||
"total_steps": (
|
||||
"Your current estimate for how many steps will be needed to complete the security audit. "
|
||||
"Adjust and increase as new security findings emerge."
|
||||
),
|
||||
"next_step_required": (
|
||||
"Set to true if you plan to continue the investigation with another step. False means you believe "
|
||||
"the security audit analysis is complete and ALL threats have been uncovered, ready for expert validation."
|
||||
),
|
||||
"findings": (
|
||||
"Summarize everything discovered in this step about security aspects of the code being audited. "
|
||||
"Include analysis of security vulnerabilities, authentication/authorization issues, input validation "
|
||||
"gaps, encryption weaknesses, configuration problems, and compliance concerns. Be specific and avoid "
|
||||
"vague language—document what you now know about the security posture and how it affects your "
|
||||
"assessment. IMPORTANT: Document both positive security findings (proper implementations, good "
|
||||
"security practices) and concerns (vulnerabilities, security gaps, compliance issues). In later "
|
||||
"steps, confirm or update past findings with additional evidence."
|
||||
),
|
||||
"files_checked": (
|
||||
"List all files (as absolute paths, do not clip or shrink file names) examined during the security "
|
||||
"audit investigation so far. Include even files ruled out or found to be unrelated, as this tracks "
|
||||
"your exploration path."
|
||||
),
|
||||
"relevant_files": (
|
||||
"For when this is the first step, please pass absolute file paths of relevant code to audit (do not clip "
|
||||
"file paths). When used for the final step, this contains a subset of files_checked (as full absolute paths) "
|
||||
"that contain code directly relevant to the security audit or contain significant security issues, patterns, "
|
||||
"or examples worth highlighting. Only list those that are directly tied to important security findings, "
|
||||
"vulnerabilities, authentication issues, or security architectural decisions. This could include "
|
||||
"authentication modules, input validation files, configuration files, or files with notable security patterns."
|
||||
),
|
||||
"relevant_context": (
|
||||
"List methods, functions, classes, or modules that are central to the security audit findings, in the "
|
||||
"format 'ClassName.methodName', 'functionName', or 'module.ClassName'. Prioritize those that contain "
|
||||
"security vulnerabilities, demonstrate security patterns, show authentication/authorization logic, or "
|
||||
"represent key security architectural decisions."
|
||||
),
|
||||
"issues_found": (
|
||||
"List of security issues identified during the investigation. Each issue should be a dictionary with "
|
||||
"'severity' (critical, high, medium, low) and 'description' fields. Include security vulnerabilities, "
|
||||
"authentication bypasses, authorization flaws, injection vulnerabilities, cryptographic weaknesses, "
|
||||
"configuration issues, compliance gaps, etc."
|
||||
),
|
||||
"confidence": (
|
||||
"Indicate your current confidence in the security audit assessment. Use: 'exploring' (starting analysis), "
|
||||
"'low' (early investigation), 'medium' (some evidence gathered), 'high' (strong evidence), 'certain' "
|
||||
"(only when the security audit is thoroughly complete and all significant security issues are identified). "
|
||||
"Do NOT use 'certain' unless the security audit is comprehensively complete, use 'high' instead not 100% "
|
||||
"sure. Using 'certain' prevents additional expert analysis."
|
||||
),
|
||||
"backtrack_from_step": (
|
||||
"If an earlier finding or assessment needs to be revised or discarded, specify the step number from which "
|
||||
"to start over. Use this to acknowledge investigative dead ends and correct the course."
|
||||
),
|
||||
"images": (
|
||||
"Optional list of absolute paths to architecture diagrams, security models, threat models, or visual "
|
||||
"references that help with security audit context. Only include if they materially assist understanding "
|
||||
"or assessment of security posture."
|
||||
),
|
||||
"security_scope": (
|
||||
"Define the security scope and application context (web app, mobile app, API, enterprise system, "
|
||||
"cloud service). Include technology stack, user types, data sensitivity, and threat landscape. "
|
||||
"This helps focus the security assessment appropriately."
|
||||
),
|
||||
"threat_level": (
|
||||
"Assess the threat level based on application context: 'low' (internal tools, low-risk data), "
|
||||
"'medium' (customer-facing, business data), 'high' (financial, healthcare, regulated industry), "
|
||||
"'critical' (payment processing, sensitive personal data). This guides prioritization."
|
||||
),
|
||||
"compliance_requirements": (
|
||||
"List applicable compliance frameworks and security standards (SOC2, PCI DSS, HIPAA, GDPR, "
|
||||
"ISO 27001, NIST). Include industry-specific requirements that affect security controls."
|
||||
),
|
||||
"audit_focus": "Primary security focus areas for this audit (owasp, compliance, infrastructure, dependencies)",
|
||||
"severity_filter": "Minimum severity level to report on the security issues found",
|
||||
}
|
||||
|
||||
|
||||
class SecauditRequest(WorkflowRequest):
|
||||
"""Request model for security audit workflow investigation steps"""
|
||||
|
||||
# Required fields for each investigation step
|
||||
step: str = Field(..., description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["step"])
|
||||
step_number: int = Field(..., description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["step_number"])
|
||||
total_steps: int = Field(..., description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["total_steps"])
|
||||
next_step_required: bool = Field(..., description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["next_step_required"])
|
||||
|
||||
# Investigation tracking fields
|
||||
findings: str = Field(..., description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["findings"])
|
||||
files_checked: list[str] = Field(
|
||||
default_factory=list, description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["files_checked"]
|
||||
)
|
||||
relevant_files: list[str] = Field(
|
||||
default_factory=list, description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["relevant_files"]
|
||||
)
|
||||
relevant_context: list[str] = Field(
|
||||
default_factory=list, description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["relevant_context"]
|
||||
)
|
||||
issues_found: list[dict] = Field(
|
||||
default_factory=list, description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["issues_found"]
|
||||
)
|
||||
confidence: Optional[str] = Field("low", description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["confidence"])
|
||||
|
||||
# Optional backtracking field
|
||||
backtrack_from_step: Optional[int] = Field(
|
||||
None, description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["backtrack_from_step"]
|
||||
)
|
||||
|
||||
# Optional images for visual context
|
||||
images: Optional[list[str]] = Field(default=None, description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["images"])
|
||||
|
||||
# Security audit-specific fields
|
||||
security_scope: Optional[str] = Field(None, description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["security_scope"])
|
||||
threat_level: Optional[Literal["low", "medium", "high", "critical"]] = Field(
|
||||
"medium", description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["threat_level"]
|
||||
)
|
||||
compliance_requirements: Optional[list[str]] = Field(
|
||||
default_factory=list, description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["compliance_requirements"]
|
||||
)
|
||||
audit_focus: Optional[Literal["owasp", "compliance", "infrastructure", "dependencies", "comprehensive"]] = Field(
|
||||
"comprehensive", description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["audit_focus"]
|
||||
)
|
||||
severity_filter: Optional[Literal["critical", "high", "medium", "low", "all"]] = Field(
|
||||
"all", description=SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["severity_filter"]
|
||||
)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_security_audit_request(self):
|
||||
"""Validate security audit request parameters"""
|
||||
# Ensure security scope is provided for comprehensive audits
|
||||
if self.step_number == 1 and not self.security_scope:
|
||||
logger.warning("Security scope not provided for security audit - defaulting to general application")
|
||||
|
||||
# Validate compliance requirements format
|
||||
if self.compliance_requirements:
|
||||
valid_compliance = {"SOC2", "PCI DSS", "HIPAA", "GDPR", "ISO 27001", "NIST", "FedRAMP", "FISMA"}
|
||||
for req in self.compliance_requirements:
|
||||
if req not in valid_compliance:
|
||||
logger.warning(f"Unknown compliance requirement: {req}")
|
||||
|
||||
return self
|
||||
|
||||
|
||||
class SecauditTool(WorkflowTool):
|
||||
"""
|
||||
Comprehensive security audit workflow tool.
|
||||
|
||||
Provides systematic security assessment through multi-step investigation
|
||||
covering OWASP Top 10, compliance requirements, and technology-specific
|
||||
security patterns. Follows established WorkflowTool patterns while adding
|
||||
security-specific capabilities.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.initial_request = None
|
||||
self.security_config = {}
|
||||
|
||||
def get_name(self) -> str:
|
||||
"""Return the unique name of the tool."""
|
||||
return "secaudit"
|
||||
|
||||
def get_description(self) -> str:
|
||||
"""Return a description of the tool."""
|
||||
return (
|
||||
"COMPREHENSIVE SECURITY AUDIT WORKFLOW - Step-by-step security assessment with expert analysis. "
|
||||
"This tool guides you through a systematic investigation process where you:\\n\\n"
|
||||
"1. Start with step 1: describe your security investigation plan\\n"
|
||||
"2. STOP and investigate code structure, patterns, and security issues\\n"
|
||||
"3. Report findings in step 2 with concrete evidence from actual code analysis\\n"
|
||||
"4. Continue investigating between each step\\n"
|
||||
"5. Track findings, relevant files, and security issues throughout\\n"
|
||||
"6. Update assessments as understanding evolves\\n"
|
||||
"7. Once investigation is complete, receive expert security analysis\\n\\n"
|
||||
"IMPORTANT: This tool enforces investigation between steps:\\n"
|
||||
"- After each call, you MUST investigate before calling again\\n"
|
||||
"- Each step must include NEW evidence from code examination\\n"
|
||||
"- No recursive calls without actual investigation work\\n"
|
||||
"- The tool will specify which step number to use next\\n"
|
||||
"- Follow the required_actions list for investigation guidance\\n\\n"
|
||||
"Perfect for: comprehensive security assessment, OWASP Top 10 analysis, compliance evaluation, "
|
||||
"vulnerability identification, threat modeling, security architecture review."
|
||||
)
|
||||
|
||||
def get_system_prompt(self) -> str:
|
||||
"""Return the system prompt for expert security analysis."""
|
||||
return SECAUDIT_PROMPT
|
||||
|
||||
def get_default_temperature(self) -> float:
|
||||
"""Return the temperature for security audit analysis"""
|
||||
return TEMPERATURE_ANALYTICAL
|
||||
|
||||
def get_model_category(self) -> "ToolModelCategory":
|
||||
"""Return the model category for security audit"""
|
||||
from tools.models import ToolModelCategory
|
||||
|
||||
return ToolModelCategory.EXTENDED_REASONING
|
||||
|
||||
def get_workflow_request_model(self) -> type:
|
||||
"""Return the workflow request model class"""
|
||||
return SecauditRequest
|
||||
|
||||
def get_tool_fields(self) -> dict[str, dict[str, Any]]:
|
||||
"""
|
||||
Get security audit tool field definitions.
|
||||
|
||||
Returns comprehensive field definitions including security-specific
|
||||
parameters while maintaining compatibility with existing workflow patterns.
|
||||
"""
|
||||
return SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS
|
||||
|
||||
def get_required_actions(self, step_number: int, confidence: str, findings: str, total_steps: int) -> list[str]:
|
||||
"""
|
||||
Provide step-specific guidance for systematic security analysis.
|
||||
|
||||
Each step focuses on specific security domains to ensure comprehensive
|
||||
coverage without missing critical security aspects.
|
||||
"""
|
||||
if step_number == 1:
|
||||
return [
|
||||
"Identify application type, technology stack, and security scope",
|
||||
"Map attack surface, entry points, and data flows",
|
||||
"Determine relevant security standards and compliance requirements",
|
||||
"Establish threat landscape and risk context for the application",
|
||||
]
|
||||
elif step_number == 2:
|
||||
return [
|
||||
"Analyze authentication mechanisms and session management",
|
||||
"Check authorization controls, access patterns, and privilege escalation risks",
|
||||
"Assess multi-factor authentication, password policies, and account security",
|
||||
"Review identity and access management implementations",
|
||||
]
|
||||
elif step_number == 3:
|
||||
return [
|
||||
"Examine input validation and sanitization mechanisms across all entry points",
|
||||
"Check for injection vulnerabilities (SQL, XSS, Command, LDAP, NoSQL)",
|
||||
"Review data encryption, sensitive data handling, and cryptographic implementations",
|
||||
"Analyze API input validation, rate limiting, and request/response security",
|
||||
]
|
||||
elif step_number == 4:
|
||||
return [
|
||||
"Conduct OWASP Top 10 (2021) systematic review across all categories",
|
||||
"Check each OWASP category methodically with specific findings and evidence",
|
||||
"Cross-reference findings with application context and technology stack",
|
||||
"Prioritize vulnerabilities based on exploitability and business impact",
|
||||
]
|
||||
elif step_number == 5:
|
||||
return [
|
||||
"Analyze third-party dependencies for known vulnerabilities and outdated versions",
|
||||
"Review configuration security, default settings, and hardening measures",
|
||||
"Check for hardcoded secrets, credentials, and sensitive information exposure",
|
||||
"Assess logging, monitoring, incident response, and security observability",
|
||||
]
|
||||
elif step_number == 6:
|
||||
return [
|
||||
"Evaluate compliance requirements and identify gaps in controls",
|
||||
"Assess business impact and risk levels of all identified findings",
|
||||
"Create prioritized remediation roadmap with timeline and effort estimates",
|
||||
"Document comprehensive security posture and recommendations",
|
||||
]
|
||||
else:
|
||||
return [
|
||||
"Continue systematic security investigation based on emerging findings",
|
||||
"Deep-dive into specific security concerns identified in previous steps",
|
||||
"Validate security hypotheses and confirm vulnerability assessments",
|
||||
]
|
||||
|
||||
def should_call_expert_analysis(self, consolidated_findings, request=None) -> bool:
|
||||
"""
|
||||
Determine when to call expert security analysis.
|
||||
|
||||
Expert analysis is triggered when the security audit has meaningful findings
|
||||
unless the user requested to skip assistant model.
|
||||
"""
|
||||
# Check if user requested to skip assistant model
|
||||
if request and not self.get_request_use_assistant_model(request):
|
||||
return False
|
||||
|
||||
# Check if we have meaningful investigation data
|
||||
return (
|
||||
len(consolidated_findings.relevant_files) > 0
|
||||
or len(consolidated_findings.findings) >= 2
|
||||
or len(consolidated_findings.issues_found) > 0
|
||||
)
|
||||
|
||||
def prepare_expert_analysis_context(self, consolidated_findings) -> str:
|
||||
"""
|
||||
Prepare comprehensive context for expert security model analysis.
|
||||
|
||||
Provides security-specific context including scope, threat level,
|
||||
compliance requirements, and systematic findings for expert validation.
|
||||
"""
|
||||
context_parts = [
|
||||
f"=== SECURITY AUDIT REQUEST ===\n{self.initial_request or 'Security audit workflow initiated'}\n=== END REQUEST ==="
|
||||
]
|
||||
|
||||
# Add investigation summary
|
||||
investigation_summary = self._build_security_audit_summary(consolidated_findings)
|
||||
context_parts.append(
|
||||
f"\n=== CLAUDE'S SECURITY INVESTIGATION ===\n{investigation_summary}\n=== END INVESTIGATION ==="
|
||||
)
|
||||
|
||||
# Add security configuration context if available
|
||||
if self.security_config:
|
||||
config_text = "\n".join(f"- {key}: {value}" for key, value in self.security_config.items() if value)
|
||||
context_parts.append(f"\n=== SECURITY CONFIGURATION ===\n{config_text}\n=== END CONFIGURATION ===")
|
||||
|
||||
# Add relevant files if available
|
||||
if consolidated_findings.relevant_files:
|
||||
files_text = "\n".join(f"- {file}" for file in consolidated_findings.relevant_files)
|
||||
context_parts.append(f"\n=== RELEVANT FILES ===\n{files_text}\n=== END FILES ===")
|
||||
|
||||
# Add relevant security elements if available
|
||||
if consolidated_findings.relevant_context:
|
||||
methods_text = "\n".join(f"- {method}" for method in consolidated_findings.relevant_context)
|
||||
context_parts.append(
|
||||
f"\n=== SECURITY-CRITICAL CODE ELEMENTS ===\n{methods_text}\n=== END CODE ELEMENTS ==="
|
||||
)
|
||||
|
||||
# Add security issues found if available
|
||||
if consolidated_findings.issues_found:
|
||||
issues_text = self._format_security_issues(consolidated_findings.issues_found)
|
||||
context_parts.append(f"\n=== SECURITY ISSUES IDENTIFIED ===\n{issues_text}\n=== END ISSUES ===")
|
||||
|
||||
# Add assessment evolution if available
|
||||
if consolidated_findings.hypotheses:
|
||||
assessments_text = "\n".join(
|
||||
f"Step {h['step']} ({h['confidence']} confidence): {h['hypothesis']}"
|
||||
for h in consolidated_findings.hypotheses
|
||||
)
|
||||
context_parts.append(f"\n=== ASSESSMENT EVOLUTION ===\n{assessments_text}\n=== END ASSESSMENTS ===")
|
||||
|
||||
# Add images if available
|
||||
if consolidated_findings.images:
|
||||
images_text = "\n".join(f"- {img}" for img in consolidated_findings.images)
|
||||
context_parts.append(
|
||||
f"\n=== VISUAL SECURITY INFORMATION ===\n{images_text}\n=== END VISUAL INFORMATION ==="
|
||||
)
|
||||
|
||||
return "\n".join(context_parts)
|
||||
|
||||
def _format_security_issues(self, issues_found: list[dict]) -> str:
|
||||
"""
|
||||
Format security issues for expert analysis.
|
||||
|
||||
Organizes security findings by severity for clear expert review.
|
||||
"""
|
||||
if not issues_found:
|
||||
return "No security issues identified during systematic investigation."
|
||||
|
||||
# Group issues by severity
|
||||
severity_groups = {"critical": [], "high": [], "medium": [], "low": []}
|
||||
|
||||
for issue in issues_found:
|
||||
severity = issue.get("severity", "low").lower()
|
||||
description = issue.get("description", "No description provided")
|
||||
if severity in severity_groups:
|
||||
severity_groups[severity].append(description)
|
||||
else:
|
||||
severity_groups["low"].append(f"[{severity.upper()}] {description}")
|
||||
|
||||
formatted_issues = []
|
||||
for severity in ["critical", "high", "medium", "low"]:
|
||||
if severity_groups[severity]:
|
||||
formatted_issues.append(f"\n{severity.upper()} SEVERITY:")
|
||||
for issue in severity_groups[severity]:
|
||||
formatted_issues.append(f" • {issue}")
|
||||
|
||||
return "\n".join(formatted_issues) if formatted_issues else "No security issues identified."
|
||||
|
||||
def _build_security_audit_summary(self, consolidated_findings) -> str:
|
||||
"""Prepare a comprehensive summary of the security audit investigation."""
|
||||
summary_parts = [
|
||||
"=== SYSTEMATIC SECURITY AUDIT INVESTIGATION SUMMARY ===",
|
||||
f"Total steps: {len(consolidated_findings.findings)}",
|
||||
f"Files examined: {len(consolidated_findings.files_checked)}",
|
||||
f"Relevant files identified: {len(consolidated_findings.relevant_files)}",
|
||||
f"Security-critical elements analyzed: {len(consolidated_findings.relevant_context)}",
|
||||
f"Security issues identified: {len(consolidated_findings.issues_found)}",
|
||||
"",
|
||||
"=== INVESTIGATION PROGRESSION ===",
|
||||
]
|
||||
|
||||
for finding in consolidated_findings.findings:
|
||||
summary_parts.append(finding)
|
||||
|
||||
return "\n".join(summary_parts)
|
||||
|
||||
def get_input_schema(self) -> dict[str, Any]:
|
||||
"""Generate input schema using WorkflowSchemaBuilder with security audit-specific overrides."""
|
||||
from .workflow.schema_builders import WorkflowSchemaBuilder
|
||||
|
||||
# Security audit workflow-specific field overrides
|
||||
secaudit_field_overrides = {
|
||||
"step": {
|
||||
"type": "string",
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["step"],
|
||||
},
|
||||
"step_number": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["step_number"],
|
||||
},
|
||||
"total_steps": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["total_steps"],
|
||||
},
|
||||
"next_step_required": {
|
||||
"type": "boolean",
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["next_step_required"],
|
||||
},
|
||||
"findings": {
|
||||
"type": "string",
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["findings"],
|
||||
},
|
||||
"files_checked": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["files_checked"],
|
||||
},
|
||||
"relevant_files": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["relevant_files"],
|
||||
},
|
||||
"confidence": {
|
||||
"type": "string",
|
||||
"enum": ["exploring", "low", "medium", "high", "certain"],
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["confidence"],
|
||||
},
|
||||
"backtrack_from_step": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["backtrack_from_step"],
|
||||
},
|
||||
"issues_found": {
|
||||
"type": "array",
|
||||
"items": {"type": "object"},
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["issues_found"],
|
||||
},
|
||||
"images": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["images"],
|
||||
},
|
||||
# Security audit-specific fields (for step 1)
|
||||
"security_scope": {
|
||||
"type": "string",
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["security_scope"],
|
||||
},
|
||||
"threat_level": {
|
||||
"type": "string",
|
||||
"enum": ["low", "medium", "high", "critical"],
|
||||
"default": "medium",
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["threat_level"],
|
||||
},
|
||||
"compliance_requirements": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["compliance_requirements"],
|
||||
},
|
||||
"audit_focus": {
|
||||
"type": "string",
|
||||
"enum": ["owasp", "compliance", "infrastructure", "dependencies", "comprehensive"],
|
||||
"default": "comprehensive",
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["audit_focus"],
|
||||
},
|
||||
"severity_filter": {
|
||||
"type": "string",
|
||||
"enum": ["critical", "high", "medium", "low", "all"],
|
||||
"default": "all",
|
||||
"description": SECAUDIT_WORKFLOW_FIELD_DESCRIPTIONS["severity_filter"],
|
||||
},
|
||||
}
|
||||
|
||||
# Use WorkflowSchemaBuilder with security audit-specific tool fields
|
||||
return WorkflowSchemaBuilder.build_schema(
|
||||
tool_specific_fields=secaudit_field_overrides,
|
||||
model_field_schema=self.get_model_field_schema(),
|
||||
auto_mode=self.is_effective_auto_mode(),
|
||||
tool_name=self.get_name(),
|
||||
)
|
||||
|
||||
# Hook method overrides for security audit-specific behavior
|
||||
|
||||
def prepare_step_data(self, request) -> dict:
|
||||
"""Map security audit-specific fields for internal processing."""
|
||||
step_data = {
|
||||
"step": request.step,
|
||||
"step_number": request.step_number,
|
||||
"findings": request.findings,
|
||||
"files_checked": request.files_checked,
|
||||
"relevant_files": request.relevant_files,
|
||||
"relevant_context": request.relevant_context,
|
||||
"issues_found": request.issues_found,
|
||||
"confidence": request.confidence,
|
||||
"hypothesis": request.findings, # Map findings to hypothesis for compatibility
|
||||
"images": request.images or [],
|
||||
}
|
||||
|
||||
# Store security-specific configuration on first step
|
||||
if request.step_number == 1:
|
||||
self.security_config = {
|
||||
"security_scope": request.security_scope,
|
||||
"threat_level": request.threat_level,
|
||||
"compliance_requirements": request.compliance_requirements,
|
||||
"audit_focus": request.audit_focus,
|
||||
"severity_filter": request.severity_filter,
|
||||
}
|
||||
|
||||
return step_data
|
||||
|
||||
def should_skip_expert_analysis(self, request, consolidated_findings) -> bool:
|
||||
"""Security audit workflow skips expert analysis when Claude has "certain" confidence."""
|
||||
return request.confidence == "certain" and not request.next_step_required
|
||||
|
||||
def store_initial_issue(self, step_description: str):
|
||||
"""Store initial request for expert analysis."""
|
||||
self.initial_request = step_description
|
||||
|
||||
def should_include_files_in_expert_prompt(self) -> bool:
|
||||
"""Include files in expert analysis for comprehensive security audit."""
|
||||
return True
|
||||
|
||||
def should_embed_system_prompt(self) -> bool:
|
||||
"""Embed system prompt in expert analysis for proper context."""
|
||||
return True
|
||||
|
||||
def get_expert_thinking_mode(self) -> str:
|
||||
"""Use high thinking mode for thorough security analysis."""
|
||||
return "high"
|
||||
|
||||
def get_expert_analysis_instruction(self) -> str:
|
||||
"""Get specific instruction for security audit expert analysis."""
|
||||
return (
|
||||
"Please provide comprehensive security analysis based on the investigation findings. "
|
||||
"Focus on identifying any remaining vulnerabilities, validating the completeness of the analysis, "
|
||||
"and providing final recommendations for security improvements, following the OWASP-based "
|
||||
"format specified in the system prompt."
|
||||
)
|
||||
|
||||
def get_completion_next_steps_message(self, expert_analysis_used: bool = False) -> str:
|
||||
"""
|
||||
Security audit-specific completion message.
|
||||
"""
|
||||
base_message = (
|
||||
"SECURITY AUDIT IS COMPLETE. You MUST now summarize and present ALL security findings organized by "
|
||||
"severity (Critical → High → Medium → Low), specific code locations with line numbers, and exact "
|
||||
"remediation steps for each vulnerability. Clearly prioritize the top 3 security issues that need "
|
||||
"immediate attention. Provide concrete, actionable guidance for each vulnerability—make it easy for "
|
||||
"developers to understand exactly what needs to be fixed and how to implement the security improvements."
|
||||
)
|
||||
|
||||
# Add expert analysis guidance only when expert analysis was actually used
|
||||
if expert_analysis_used:
|
||||
expert_guidance = self.get_expert_analysis_guidance()
|
||||
if expert_guidance:
|
||||
return f"{base_message}\n\n{expert_guidance}"
|
||||
|
||||
return base_message
|
||||
|
||||
def get_expert_analysis_guidance(self) -> str:
|
||||
"""
|
||||
Provide specific guidance for handling expert analysis in security audits.
|
||||
"""
|
||||
return (
|
||||
"IMPORTANT: Analysis from an assistant model has been provided above. You MUST critically evaluate and validate "
|
||||
"the expert security findings rather than accepting them blindly. Cross-reference the expert analysis with "
|
||||
"your own investigation findings, verify that suggested security improvements are appropriate for this "
|
||||
"application's context and threat model, and ensure recommendations align with the project's security requirements. "
|
||||
"Present a synthesis that combines your systematic security review with validated expert insights, clearly "
|
||||
"distinguishing between vulnerabilities you've independently confirmed and additional insights from expert analysis."
|
||||
)
|
||||
|
||||
def get_step_guidance_message(self, request) -> str:
|
||||
"""
|
||||
Security audit-specific step guidance with detailed investigation instructions.
|
||||
"""
|
||||
step_guidance = self.get_security_audit_step_guidance(request.step_number, request.confidence, request)
|
||||
return step_guidance["next_steps"]
|
||||
|
||||
def get_security_audit_step_guidance(self, step_number: int, confidence: str, request) -> dict[str, Any]:
|
||||
"""
|
||||
Provide step-specific guidance for security audit workflow.
|
||||
"""
|
||||
# Generate the next steps instruction based on required actions
|
||||
required_actions = self.get_required_actions(step_number, confidence, request.findings, request.total_steps)
|
||||
|
||||
if step_number == 1:
|
||||
next_steps = (
|
||||
f"MANDATORY: DO NOT call the {self.get_name()} tool again immediately. You MUST first examine "
|
||||
f"the code files thoroughly using appropriate tools. CRITICAL AWARENESS: You need to understand "
|
||||
f"the security landscape, identify potential vulnerabilities across OWASP Top 10 categories, "
|
||||
f"and look for authentication flaws, injection points, cryptographic issues, and authorization bypasses. "
|
||||
f"Use file reading tools, security analysis, and systematic examination to gather comprehensive information. "
|
||||
f"Only call {self.get_name()} again AFTER completing your security investigation. When you call "
|
||||
f"{self.get_name()} next time, use step_number: {step_number + 1} and report specific "
|
||||
f"files examined, vulnerabilities found, and security assessments discovered."
|
||||
)
|
||||
elif confidence in ["exploring", "low"]:
|
||||
next_steps = (
|
||||
f"STOP! Do NOT call {self.get_name()} again yet. Based on your findings, you've identified areas that need "
|
||||
f"deeper security analysis. MANDATORY ACTIONS before calling {self.get_name()} step {step_number + 1}:\n"
|
||||
+ "\n".join(f"{i+1}. {action}" for i, action in enumerate(required_actions))
|
||||
+ f"\n\nOnly call {self.get_name()} again with step_number: {step_number + 1} AFTER "
|
||||
+ "completing these security audit tasks."
|
||||
)
|
||||
elif confidence in ["medium", "high"]:
|
||||
next_steps = (
|
||||
f"WAIT! Your security audit needs final verification. DO NOT call {self.get_name()} immediately. REQUIRED ACTIONS:\n"
|
||||
+ "\n".join(f"{i+1}. {action}" for i, action in enumerate(required_actions))
|
||||
+ f"\n\nREMEMBER: Ensure you have identified all significant vulnerabilities across all severity levels and "
|
||||
f"verified the completeness of your security review. Document findings with specific file references and "
|
||||
f"line numbers where applicable, then call {self.get_name()} with step_number: {step_number + 1}."
|
||||
)
|
||||
else:
|
||||
next_steps = (
|
||||
f"PAUSE SECURITY AUDIT. Before calling {self.get_name()} step {step_number + 1}, you MUST examine more code thoroughly. "
|
||||
+ "Required: "
|
||||
+ ", ".join(required_actions[:2])
|
||||
+ ". "
|
||||
+ f"Your next {self.get_name()} call (step_number: {step_number + 1}) must include "
|
||||
f"NEW evidence from actual security analysis, not just theories. NO recursive {self.get_name()} calls "
|
||||
f"without investigation work!"
|
||||
)
|
||||
|
||||
return {"next_steps": next_steps}
|
||||
|
||||
def customize_workflow_response(self, response_data: dict, request) -> dict:
|
||||
"""
|
||||
Customize response to match security audit workflow format.
|
||||
"""
|
||||
# Store initial request on first step
|
||||
if request.step_number == 1:
|
||||
self.initial_request = request.step
|
||||
# Store security configuration for expert analysis
|
||||
if request.relevant_files:
|
||||
self.security_config = {
|
||||
"relevant_files": request.relevant_files,
|
||||
"security_scope": request.security_scope,
|
||||
"threat_level": request.threat_level,
|
||||
"compliance_requirements": request.compliance_requirements,
|
||||
"audit_focus": request.audit_focus,
|
||||
"severity_filter": request.severity_filter,
|
||||
}
|
||||
|
||||
# Convert generic status names to security audit-specific ones
|
||||
tool_name = self.get_name()
|
||||
status_mapping = {
|
||||
f"{tool_name}_in_progress": "security_audit_in_progress",
|
||||
f"pause_for_{tool_name}": "pause_for_security_audit",
|
||||
f"{tool_name}_required": "security_audit_required",
|
||||
f"{tool_name}_complete": "security_audit_complete",
|
||||
}
|
||||
|
||||
if response_data["status"] in status_mapping:
|
||||
response_data["status"] = status_mapping[response_data["status"]]
|
||||
|
||||
# Rename status field to match security audit workflow
|
||||
if f"{tool_name}_status" in response_data:
|
||||
response_data["security_audit_status"] = response_data.pop(f"{tool_name}_status")
|
||||
# Add security audit-specific status fields
|
||||
response_data["security_audit_status"]["vulnerabilities_by_severity"] = {}
|
||||
for issue in self.consolidated_findings.issues_found:
|
||||
severity = issue.get("severity", "unknown")
|
||||
if severity not in response_data["security_audit_status"]["vulnerabilities_by_severity"]:
|
||||
response_data["security_audit_status"]["vulnerabilities_by_severity"][severity] = 0
|
||||
response_data["security_audit_status"]["vulnerabilities_by_severity"][severity] += 1
|
||||
response_data["security_audit_status"]["audit_confidence"] = self.get_request_confidence(request)
|
||||
|
||||
# Map complete_secaudit to complete_security_audit
|
||||
if f"complete_{tool_name}" in response_data:
|
||||
response_data["complete_security_audit"] = response_data.pop(f"complete_{tool_name}")
|
||||
|
||||
# Map the completion flag to match security audit workflow
|
||||
if f"{tool_name}_complete" in response_data:
|
||||
response_data["security_audit_complete"] = response_data.pop(f"{tool_name}_complete")
|
||||
|
||||
return response_data
|
||||
|
||||
# Override inheritance hooks for security audit-specific behavior
|
||||
|
||||
def get_completion_status(self) -> str:
|
||||
"""Security audit tools use audit-specific status."""
|
||||
return "security_analysis_complete"
|
||||
|
||||
def get_completion_data_key(self) -> str:
|
||||
"""Security audit uses 'complete_security_audit' key."""
|
||||
return "complete_security_audit"
|
||||
|
||||
def get_final_analysis_from_request(self, request):
|
||||
"""Security audit tools use 'findings' field."""
|
||||
return request.findings
|
||||
|
||||
def get_confidence_level(self, request) -> str:
|
||||
"""Security audit tools use 'certain' for high confidence."""
|
||||
return "certain"
|
||||
|
||||
def get_completion_message(self) -> str:
|
||||
"""Security audit-specific completion message."""
|
||||
return (
|
||||
"Security audit complete with CERTAIN confidence. You have identified all significant vulnerabilities "
|
||||
"and provided comprehensive security analysis. MANDATORY: Present the user with the complete security audit results "
|
||||
"categorized by severity, and IMMEDIATELY proceed with implementing the highest priority security fixes "
|
||||
"or provide specific guidance for vulnerability remediation. Focus on actionable security recommendations."
|
||||
)
|
||||
|
||||
def get_skip_reason(self) -> str:
|
||||
"""Security audit-specific skip reason."""
|
||||
return "Claude completed comprehensive security audit with full confidence"
|
||||
|
||||
def get_skip_expert_analysis_status(self) -> str:
|
||||
"""Security audit-specific expert analysis skip status."""
|
||||
return "skipped_due_to_certain_audit_confidence"
|
||||
|
||||
def prepare_work_summary(self) -> str:
|
||||
"""Security audit-specific work summary."""
|
||||
return self._build_security_audit_summary(self.consolidated_findings)
|
||||
|
||||
def get_request_model(self):
|
||||
"""Return the request model for this tool"""
|
||||
return SecauditRequest
|
||||
|
||||
async def prepare_prompt(self, request: SecauditRequest) -> str:
|
||||
"""Not used - workflow tools use execute_workflow()."""
|
||||
return "" # Workflow tools use execute_workflow() directly
|
||||
Reference in New Issue
Block a user