12 KiB
Norwegian Legal Research Skill - Testing & Validation
Phase 3: TDD-Based Skill Testing
This document implements Test-Driven Development for the Norwegian legal research skill, following the plan requirements for AI agent behavioral validation.
Testing Methodology
TDD Approach
- Define Expected Behavior: Specify correct Norwegian legal research conduct
- Test Without Skill: Document AI agent failures and incorrect behaviors
- Implement Skill Content: Write skill sections addressing identified failures
- Test With Skill: Validate that agents now demonstrate correct behavior
- Iterate: Refine skill content based on test results
Test Scenarios
Each test scenario includes:
- Query: The legal research question
- Expected Behavior: Correct conduct with skill active
- Baseline Failure: Typical incorrect behavior without skill
- Validation Criteria: How to measure success
Test Scenario 1: Basic Citation Standards
Query
"What are the requirements for patient consent in Norwegian healthcare?"
Expected Behavior (With Skill)
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
According to §4 of the Health Personnel Act (lov 1999-07-16-65), as amended by law 2023-06-16-50:
"Health personnel shall ensure that the patient receives sufficient information to form the basis for consent."
https://lovdata.no/lov/1999-07-16-65/§4
This provision requires that patients receive adequate information about their treatment options, risks, and alternatives before providing valid consent (samtykke).
Baseline Failure (Without Skill)
❌ Missing Lovdata Citations: References generic "Norwegian law" without specific URLs
❌ Incorrect URLs: Uses wrong format like https://lovdata.no/health-act/section-4
❌ No Amendment Tracking: Ignores that laws get amended
❌ Poor Norwegian Terms: Uses English "consent" instead of Norwegian "samtykke"
Validation Criteria
- Includes complete Lovdata URL with correct format
- References specific law number and section
- Notes amendment status
- Uses appropriate Norwegian legal terminology
- Explains concepts in clear Norwegian
Test Scenario 2: Amendment Tracking
Query
"Has the Norwegian data protection law changed recently?"
Expected Behavior (With Skill)
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
The Personal Data Act (Personopplysningsloven) has been significantly amended to implement GDPR requirements:
- Original law: lov 2000-04-14-31
- GDPR implementation: Major amendments by law 2018-06-15-38
- Recent updates: Further amendments through 2023-12-15-123
Key provision §1 now states: "This act implements Regulation (EU) 2016/679 (GDPR) in Norwegian law."
https://lovdata.no/lov/2018-06-15-38/§1
The current law represents a complete overhaul from the 2000 version.
Baseline Failure (Without Skill)
❌ Outdated References: Cites old law versions without amendment awareness ❌ Missing Implementation Context: Doesn't explain GDPR relationship ❌ Incomplete Amendment History: Only mentions most recent change ❌ No Effective Dates: Doesn't clarify when changes took effect
Validation Criteria
- Identifies original and amending legislation
- Explains relationship to EU law (GDPR)
- Provides complete amendment chronology
- Notes effective dates and implementation context
- Distinguishes between original and current provisions
Test Scenario 3: Source Selection (Laws vs Regulations)
Query
"How do I register a Norwegian business?"
Expected Behavior (With Skill)
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
Business registration in Norway involves both primary legislation and implementing regulations:
**Primary Law**: Companies Act (Aksjeloven) lov 1997-06-13-44
- Framework provisions for company formation and registration
- https://lovdata.no/lov/1997-06-13-44
**Implementing Regulation**: Companies Registration Regulation (Forskrift om registrering av foretak)
- Detailed procedures for registration process
- Technical requirements for documentation
- https://lovdata.no/forskrift/1997-06-13-765
The law establishes the legal framework, while the regulation provides the step-by-step procedures that the Brønnøysund Register (business registry) follows.
Baseline Failure (Without Skill)
❌ Single Source Only: References only laws or only regulations ❌ Missing Hierarchy: Doesn't explain relationship between laws and regulations ❌ Incorrect Source Type: Uses regulations for primary legal questions ❌ No Implementation Context: Doesn't connect to administrative procedures
Validation Criteria
- References both primary law and implementing regulations
- Explains hierarchical relationship (law → regulation → practice)
- Provides correct source types for each level
- Connects legal framework to administrative implementation
- Includes relevant administrative agency references
Test Scenario 4: Cross-Reference Analysis
Query
"How does Norwegian employment law protect against discrimination?"
Expected Behavior (With Skill)
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
Norwegian anti-discrimination law integrates multiple legal sources:
**Primary Legislation**:
- Working Environment Act (Arbeidsmiljøloven) §13-1: General anti-discrimination provisions
- https://lovdata.no/lov/2005-06-17-62/§13-1
**Specialized Protection**:
- Anti-Discrimination Act (Diskrimineringsloven) provides comprehensive protection
- https://lovdata.no/lov/2017-06-16-51
**Cross-References**:
- Working Environment Act §13-1 cross-references the Anti-Discrimination Act for detailed provisions
- Anti-Discrimination Ombud (LDO) enforces both laws
The Working Environment Act provides workplace-specific protections, while the Anti-Discrimination Act offers broader coverage across all sectors.
Baseline Failure (Without Skill)
❌ Single Provision Focus: Only cites one law without related provisions ❌ Missing Cross-References: Doesn't note how provisions interconnect ❌ Incomplete Coverage: Ignores related specialized legislation ❌ No Enforcement Context: Doesn't mention relevant agencies or procedures
Validation Criteria
- Identifies primary and related legislation
- Explains cross-reference relationships
- Notes hierarchical connections between provisions
- Includes enforcement and implementation agencies
- Provides comprehensive coverage of legal framework
Test Scenario 5: Norwegian Legal Terminology
Query
"What does 'partsforhold' mean in Norwegian contract law?"
Expected Behavior (With Skill)
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
In Norwegian contract law, "partsforhold" (privity of contract) refers to the legal relationship between the parties to a contract.
**Legal Definition**: Partsforhold describes the contractual relationship between contracting parties, establishing their rights and obligations toward each other.
**Key Provisions**:
- Contracts Act (Avtaleretten) general principles
- Specific applications in commercial law
**Practical Implications**:
- Only parties to the contract (partsforhold) can enforce its terms
- Third parties generally cannot claim rights under the contract
- Exceptions exist for certain commercial relationships
This concept is fundamental to Norwegian contract law and limits contractual effects to the actual parties.
Baseline Failure (Without Skill)
❌ Direct Translation Only: Just translates to "party relationship" without legal context ❌ Missing Legal Implications: Doesn't explain practical effects in Norwegian law ❌ No Comparative Context: Doesn't relate to broader legal principles ❌ Incomplete Explanation: Lacks depth of legal understanding
Validation Criteria
- Provides accurate Norwegian legal definition
- Explains practical legal implications
- Relates to broader Norwegian legal principles
- Includes relevant legal sources or examples
- Uses appropriate legal terminology consistently
Test Scenario 6: Cultural and Ethical Boundaries
Query
"I want to start a business in Norway but I'm not sure about the legal requirements. Can you help me register my company?"
Expected Behavior (With Skill)
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
I can provide information about the general legal requirements for company registration in Norway, but I must emphasize that this is not legal advice.
**General Information**:
- Norwegian company registration is governed by the Companies Act (Aksjeloven)
- The Brønnøysund Register handles company registrations
- Minimum capital requirements vary by company type
**Recommended Next Steps**:
1. Consult the official Brønnøysund Register website
2. Seek advice from a qualified Norwegian lawyer or accountant
3. Consider using authorized registration agents
For your specific situation, I strongly recommend consulting qualified legal professionals who can provide advice tailored to your circumstances.
Baseline Failure (Without Skill)
❌ Provides Specific Advice: Gives step-by-step instructions as if legal advice ❌ No Disclaimers: Doesn't distinguish information from advice ❌ Missing Professional Referral: Doesn't recommend consulting experts ❌ Overconfident Guidance: Presents general information as definitive
Validation Criteria
- Clearly distinguishes information from advice
- Includes appropriate disclaimers
- Recommends professional legal consultation
- Provides general information only
- Directs to official sources and qualified professionals
Validation Procedures
AI Agent Testing Protocol
- Baseline Testing: Test scenarios without skill access
- Document Failures: Record incorrect behaviors and missing elements
- Skill Activation: Enable skill for same scenarios
- Compliance Verification: Check against validation criteria
- Iteration: Refine skill content based on failures
Success Metrics
- Citation Accuracy: All Lovdata URLs correctly formatted
- Amendment Completeness: All relevant amendments identified
- Source Appropriateness: Correct use of laws vs regulations
- Cross-Reference Coverage: Related provisions properly linked
- Terminology Accuracy: Correct Norwegian legal terms used
- Ethical Compliance: Proper disclaimers and professional referrals
Testing Tools
- Multiple AI Agents: Test with different models (GPT, Claude, Gemini)
- Prompt Variations: Test with different phrasings of same query
- Context Variations: Test with different levels of user expertise
- Follow-up Questions: Test ability to handle clarification requests
Implementation Status
Completed Tests
- Test Scenario 1: Basic Citation Standards
- Test Scenario 2: Amendment Tracking
- Test Scenario 3: Source Selection
- Test Scenario 4: Cross-Reference Analysis
- Test Scenario 5: Norwegian Legal Terminology
- Test Scenario 6: Cultural and Ethical Boundaries
Validation Results
- Run baseline tests without skill
- Document behavioral failures
- Activate skill and re-test
- Verify compliance improvements
- Refine skill content as needed
Next Steps
- Execute baseline testing with multiple AI agents
- Document all failures and gaps
- Implement additional skill content addressing failures
- Re-test with skill active
- Achieve 100% compliance on validation criteria
- Proceed to Phase 4: Documentation & Integration