torbjorn/my-pal-mcp-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Steer the LLMs away from being overtly assertive in trying to discover non-existent bugs during a debug investigation session

Browse Source 
Add a no_bug_found response
This commit is contained in:
Fahad
2025-06-17 17:01:46 +04:00
parent 044a8621a3
commit 3667ed3a43
3 changed files with 47 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
systemprompts/debug_prompt.py
View File

@@ -47,13 +47,27 @@ You MUST respond with a properly formatted JSON object following this exact sche
Do NOT include any text before or after the JSON. The response must be valid JSON only.
IF MORE INFORMATION IS NEEDED:
If you lack critical information to proceed, respond with:
If you lack critical information to proceed, you MUST only respond with the following:
{
"status": "clarification_required",
"question": "<your brief question>",
"files_needed": ["[file name here]", "[or some folder/]"]
}
IF NO BUG FOUND AFTER THOROUGH INVESTIGATION:
If after a very thorough investigation, no concrete evidence of a bug is found correlating to reported symptoms, you
MUST only respond with the following:
{
"status": "no_bug_found",
"summary": "<summary of what was thoroughly investigated>",
"investigation_steps": ["<step 1>", "<step 2>", "..."],
"areas_examined": ["<code areas>", "<potential failure points>", "..."],
"confidence_level": "High|Medium|Low",
"alternative_explanations": ["<possible misunderstanding>", "<user expectation mismatch>", "..."],
"recommended_questions": ["<question 1 to clarify the issue>", "<question 2 to gather more context>", "..."],
"next_steps": ["<suggested actions to better understand the reported issue>"]
}
FOR COMPLETE ANALYSIS:
{
"status": "analysis_complete",
@@ -105,6 +119,10 @@ CRITICAL DEBUGGING PRINCIPLES:
4. Document your investigation process systematically for future reference
5. Rank hypotheses by likelihood based on evidence from the actual code and logs provided
6. Always include specific file:line references for exact locations of issues
7. CRITICAL: If Claude's investigation finds no concrete evidence of a bug correlating to reported symptoms,
you should consider that the reported issue may not actually exist, may be a misunderstanding, or may be
conflated with something else entirely. In such cases, recommend gathering more information from the user
through targeted questioning rather than continuing to hunt for non-existent bugs
PRECISE LOCATION REFERENCES:
When you identify specific code locations for hypotheses, include optional precision fields:

11
tools/debug.py
View File

@@ -17,7 +17,7 @@ from .base import BaseTool, ToolRequest
# Field descriptions to avoid duplication between Pydantic and JSON schema
DEBUG_FIELD_DESCRIPTIONS = {
"prompt": (
"Issue description. Include what you can provide: "
"Claud - you MUST first think deep. Issue description. Include what you can provide: "
"error messages, symptoms, when it occurs, steps to reproduce, environment details, "
"recent changes, and any other relevant information. Mention any previous attempts at fixing this issue, "
"including any past fix that was in place but has now regressed. "
@@ -27,13 +27,18 @@ DEBUG_FIELD_DESCRIPTIONS = {
"Claude MUST maintain detailed investigation notes in a DEBUGGING_{issue_description}.md file within the project folder, "
"updating it as it performs step-by-step analysis of the code, trying to determine the actual root cause and understanding how a minimal, appropriate fix can be found. "
"This file MUST contain functions, methods, files visited OR determined to be part of the problem. Claude MUST update this and remove any references that it finds to be irrelevant during its investigation. "
"CRITICAL: If after thorough investigation Claude has very high confidence that NO BUG EXISTS that correlates to the reported symptoms, "
"Claude should consider the possibility that the reported issue may not actually be present, may be a misunderstanding, or may be conflated with something else entirely. "
"In such cases, Claude should gather more information from the user through targeted questioning rather than continue hunting for non-existent bugs. "
"Once complete, Claude MUST provide Zen's debug tool with this file passed into the files parameter. "
"It is ESSENTIAL that this detailed work is performed by Claude before sharing all the relevant details with its development assistant. This will greatly help in zeroing in on the root cause."
),
"findings": (
"Claude MUST first perform its own investigation, gather its findings and analysis. Include: steps taken to analyze the issue, "
"code patterns discovered, initial hypotheses formed, any relevant classes/functions/methods examined, "
"and any preliminary conclusions. This provides context for the assistant model's analysis."
"and any preliminary conclusions. If investigation yields no concrete evidence of a bug correlating to the reported symptoms, "
"Claude should clearly state this finding and consider that the issue may not exist as described. "
"This provides context for the assistant model's analysis."
),
"files": (
"Essential files for debugging - ONLY include files that are directly related to the issue, "
@@ -70,6 +75,7 @@ class DebugIssueTool(BaseTool):
"DEBUG & ROOT CAUSE ANALYSIS - Expert debugging for complex issues with systematic investigation support. "
"Use this when you need to debug code, find out why something is failing, identify root causes, "
"trace errors, or diagnose issues. "
"MANDATORY: Claud you MUST first think deep and follow these instructions when using this tool"
"SYSTEMATIC INVESTIGATION WORKFLOW: "
"Claude MUST begin by thinking hard and performing a thorough investigation using a systematic approach. "
"First understand the issue, find the code that may be causing it or code that is breaking, as well as any related code that could have caused this as a side effect. "
@@ -92,6 +98,7 @@ class DebugIssueTool(BaseTool):
"- Use tracer tool for complex method interactions and dependencies if and as needed but continue with the investigation after using it "
"- Test hypotheses against actual code and logs and confirm the idea holds "
"- Document everything systematically "
"- CRITICAL: If investigation yields no concrete evidence of a bug, consider that the reported issue may not exist as described and gather more information through questioning "
"ESSENTIAL FILES ONLY: Include only files (documents, code etc) directly related to the issue. "
"Focus on quality over quantity for assistant model analysis. "
"STRUCTURED OUTPUT: Assistant models return JSON responses with hypothesis "

19
tools/models.py
View File

@@ -45,6 +45,7 @@ class ToolOutput(BaseModel):
"resend_prompt",
"code_too_large",
"continuation_available",
"no_bug_found",
] = "success"
content: Optional[str] = Field(None, description="The main content/response from the tool")
content_type: Literal["text", "markdown", "json"] = "text"
@@ -342,6 +343,23 @@ class DebugAnalysisComplete(BaseModel):
)
class NoBugFound(BaseModel):
"""Response when thorough investigation finds no concrete evidence of a bug"""
status: Literal["no_bug_found"] = "no_bug_found"
summary: str = Field(..., description="Summary of what was thoroughly investigated")
investigation_steps: list[str] = Field(..., description="Steps taken during the investigation")
areas_examined: list[str] = Field(..., description="Code areas and potential failure points examined")
confidence_level: Literal["High", "Medium", "Low"] = Field(
..., description="Confidence level in the no-bug finding"
)
alternative_explanations: list[str] = Field(
..., description="Possible alternative explanations for reported symptoms"
)
recommended_questions: list[str] = Field(..., description="Questions to clarify the issue with the user")
next_steps: list[str] = Field(..., description="Suggested actions to better understand the reported issue")
# Registry mapping status strings to their corresponding Pydantic models
SPECIAL_STATUS_MODELS = {
"clarification_required": ClarificationRequest,
@@ -354,4 +372,5 @@ SPECIAL_STATUS_MODELS = {
"resend_prompt": ResendPromptRequest,
"code_too_large": CodeTooLargeRequest,
"analysis_complete": DebugAnalysisComplete,
"no_bug_found": NoBugFound,
}