286 lines
12 KiB
Markdown
286 lines
12 KiB
Markdown
# Norwegian Legal Research Skill - Testing & Validation
|
|
|
|
## Phase 3: TDD-Based Skill Testing
|
|
|
|
This document implements Test-Driven Development for the Norwegian legal research skill, following the plan requirements for AI agent behavioral validation.
|
|
|
|
## Testing Methodology
|
|
|
|
### TDD Approach
|
|
1. **Define Expected Behavior**: Specify correct Norwegian legal research conduct
|
|
2. **Test Without Skill**: Document AI agent failures and incorrect behaviors
|
|
3. **Implement Skill Content**: Write skill sections addressing identified failures
|
|
4. **Test With Skill**: Validate that agents now demonstrate correct behavior
|
|
5. **Iterate**: Refine skill content based on test results
|
|
|
|
### Test Scenarios
|
|
Each test scenario includes:
|
|
- **Query**: The legal research question
|
|
- **Expected Behavior**: Correct conduct with skill active
|
|
- **Baseline Failure**: Typical incorrect behavior without skill
|
|
- **Validation Criteria**: How to measure success
|
|
|
|
## Test Scenario 1: Basic Citation Standards
|
|
|
|
### Query
|
|
"What are the requirements for patient consent in Norwegian healthcare?"
|
|
|
|
### Expected Behavior (With Skill)
|
|
```
|
|
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
|
|
|
|
According to §4 of the Health Personnel Act (lov 1999-07-16-65), as amended by law 2023-06-16-50:
|
|
"Health personnel shall ensure that the patient receives sufficient information to form the basis for consent."
|
|
|
|
https://lovdata.no/lov/1999-07-16-65/§4
|
|
|
|
This provision requires that patients receive adequate information about their treatment options, risks, and alternatives before providing valid consent (samtykke).
|
|
```
|
|
|
|
### Baseline Failure (Without Skill)
|
|
❌ **Missing Lovdata Citations**: References generic "Norwegian law" without specific URLs
|
|
❌ **Incorrect URLs**: Uses wrong format like `https://lovdata.no/health-act/section-4`
|
|
❌ **No Amendment Tracking**: Ignores that laws get amended
|
|
❌ **Poor Norwegian Terms**: Uses English "consent" instead of Norwegian "samtykke"
|
|
|
|
### Validation Criteria
|
|
- [ ] Includes complete Lovdata URL with correct format
|
|
- [ ] References specific law number and section
|
|
- [ ] Notes amendment status
|
|
- [ ] Uses appropriate Norwegian legal terminology
|
|
- [ ] Explains concepts in clear Norwegian
|
|
|
|
## Test Scenario 2: Amendment Tracking
|
|
|
|
### Query
|
|
"Has the Norwegian data protection law changed recently?"
|
|
|
|
### Expected Behavior (With Skill)
|
|
```
|
|
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
|
|
|
|
The Personal Data Act (Personopplysningsloven) has been significantly amended to implement GDPR requirements:
|
|
|
|
- Original law: lov 2000-04-14-31
|
|
- GDPR implementation: Major amendments by law 2018-06-15-38
|
|
- Recent updates: Further amendments through 2023-12-15-123
|
|
|
|
Key provision §1 now states: "This act implements Regulation (EU) 2016/679 (GDPR) in Norwegian law."
|
|
|
|
https://lovdata.no/lov/2018-06-15-38/§1
|
|
|
|
The current law represents a complete overhaul from the 2000 version.
|
|
```
|
|
|
|
### Baseline Failure (Without Skill)
|
|
❌ **Outdated References**: Cites old law versions without amendment awareness
|
|
❌ **Missing Implementation Context**: Doesn't explain GDPR relationship
|
|
❌ **Incomplete Amendment History**: Only mentions most recent change
|
|
❌ **No Effective Dates**: Doesn't clarify when changes took effect
|
|
|
|
### Validation Criteria
|
|
- [ ] Identifies original and amending legislation
|
|
- [ ] Explains relationship to EU law (GDPR)
|
|
- [ ] Provides complete amendment chronology
|
|
- [ ] Notes effective dates and implementation context
|
|
- [ ] Distinguishes between original and current provisions
|
|
|
|
## Test Scenario 3: Source Selection (Laws vs Regulations)
|
|
|
|
### Query
|
|
"How do I register a Norwegian business?"
|
|
|
|
### Expected Behavior (With Skill)
|
|
```
|
|
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
|
|
|
|
Business registration in Norway involves both primary legislation and implementing regulations:
|
|
|
|
**Primary Law**: Companies Act (Aksjeloven) lov 1997-06-13-44
|
|
- Framework provisions for company formation and registration
|
|
- https://lovdata.no/lov/1997-06-13-44
|
|
|
|
**Implementing Regulation**: Companies Registration Regulation (Forskrift om registrering av foretak)
|
|
- Detailed procedures for registration process
|
|
- Technical requirements for documentation
|
|
- https://lovdata.no/forskrift/1997-06-13-765
|
|
|
|
The law establishes the legal framework, while the regulation provides the step-by-step procedures that the Brønnøysund Register (business registry) follows.
|
|
```
|
|
|
|
### Baseline Failure (Without Skill)
|
|
❌ **Single Source Only**: References only laws or only regulations
|
|
❌ **Missing Hierarchy**: Doesn't explain relationship between laws and regulations
|
|
❌ **Incorrect Source Type**: Uses regulations for primary legal questions
|
|
❌ **No Implementation Context**: Doesn't connect to administrative procedures
|
|
|
|
### Validation Criteria
|
|
- [ ] References both primary law and implementing regulations
|
|
- [ ] Explains hierarchical relationship (law → regulation → practice)
|
|
- [ ] Provides correct source types for each level
|
|
- [ ] Connects legal framework to administrative implementation
|
|
- [ ] Includes relevant administrative agency references
|
|
|
|
## Test Scenario 4: Cross-Reference Analysis
|
|
|
|
### Query
|
|
"How does Norwegian employment law protect against discrimination?"
|
|
|
|
### Expected Behavior (With Skill)
|
|
```
|
|
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
|
|
|
|
Norwegian anti-discrimination law integrates multiple legal sources:
|
|
|
|
**Primary Legislation**:
|
|
- Working Environment Act (Arbeidsmiljøloven) §13-1: General anti-discrimination provisions
|
|
- https://lovdata.no/lov/2005-06-17-62/§13-1
|
|
|
|
**Specialized Protection**:
|
|
- Anti-Discrimination Act (Diskrimineringsloven) provides comprehensive protection
|
|
- https://lovdata.no/lov/2017-06-16-51
|
|
|
|
**Cross-References**:
|
|
- Working Environment Act §13-1 cross-references the Anti-Discrimination Act for detailed provisions
|
|
- Anti-Discrimination Ombud (LDO) enforces both laws
|
|
|
|
The Working Environment Act provides workplace-specific protections, while the Anti-Discrimination Act offers broader coverage across all sectors.
|
|
```
|
|
|
|
### Baseline Failure (Without Skill)
|
|
❌ **Single Provision Focus**: Only cites one law without related provisions
|
|
❌ **Missing Cross-References**: Doesn't note how provisions interconnect
|
|
❌ **Incomplete Coverage**: Ignores related specialized legislation
|
|
❌ **No Enforcement Context**: Doesn't mention relevant agencies or procedures
|
|
|
|
### Validation Criteria
|
|
- [ ] Identifies primary and related legislation
|
|
- [ ] Explains cross-reference relationships
|
|
- [ ] Notes hierarchical connections between provisions
|
|
- [ ] Includes enforcement and implementation agencies
|
|
- [ ] Provides comprehensive coverage of legal framework
|
|
|
|
## Test Scenario 5: Norwegian Legal Terminology
|
|
|
|
### Query
|
|
"What does 'partsforhold' mean in Norwegian contract law?"
|
|
|
|
### Expected Behavior (With Skill)
|
|
```
|
|
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
|
|
|
|
In Norwegian contract law, "partsforhold" (privity of contract) refers to the legal relationship between the parties to a contract.
|
|
|
|
**Legal Definition**: Partsforhold describes the contractual relationship between contracting parties, establishing their rights and obligations toward each other.
|
|
|
|
**Key Provisions**:
|
|
- Contracts Act (Avtaleretten) general principles
|
|
- Specific applications in commercial law
|
|
|
|
**Practical Implications**:
|
|
- Only parties to the contract (partsforhold) can enforce its terms
|
|
- Third parties generally cannot claim rights under the contract
|
|
- Exceptions exist for certain commercial relationships
|
|
|
|
This concept is fundamental to Norwegian contract law and limits contractual effects to the actual parties.
|
|
```
|
|
|
|
### Baseline Failure (Without Skill)
|
|
❌ **Direct Translation Only**: Just translates to "party relationship" without legal context
|
|
❌ **Missing Legal Implications**: Doesn't explain practical effects in Norwegian law
|
|
❌ **No Comparative Context**: Doesn't relate to broader legal principles
|
|
❌ **Incomplete Explanation**: Lacks depth of legal understanding
|
|
|
|
### Validation Criteria
|
|
- [ ] Provides accurate Norwegian legal definition
|
|
- [ ] Explains practical legal implications
|
|
- [ ] Relates to broader Norwegian legal principles
|
|
- [ ] Includes relevant legal sources or examples
|
|
- [ ] Uses appropriate legal terminology consistently
|
|
|
|
## Test Scenario 6: Cultural and Ethical Boundaries
|
|
|
|
### Query
|
|
"I want to start a business in Norway but I'm not sure about the legal requirements. Can you help me register my company?"
|
|
|
|
### Expected Behavior (With Skill)
|
|
```
|
|
I'm using the norwegian-legal-research skill to ensure proper legal research methodology.
|
|
|
|
I can provide information about the general legal requirements for company registration in Norway, but I must emphasize that this is not legal advice.
|
|
|
|
**General Information**:
|
|
- Norwegian company registration is governed by the Companies Act (Aksjeloven)
|
|
- The Brønnøysund Register handles company registrations
|
|
- Minimum capital requirements vary by company type
|
|
|
|
**Recommended Next Steps**:
|
|
1. Consult the official Brønnøysund Register website
|
|
2. Seek advice from a qualified Norwegian lawyer or accountant
|
|
3. Consider using authorized registration agents
|
|
|
|
For your specific situation, I strongly recommend consulting qualified legal professionals who can provide advice tailored to your circumstances.
|
|
```
|
|
|
|
### Baseline Failure (Without Skill)
|
|
❌ **Provides Specific Advice**: Gives step-by-step instructions as if legal advice
|
|
❌ **No Disclaimers**: Doesn't distinguish information from advice
|
|
❌ **Missing Professional Referral**: Doesn't recommend consulting experts
|
|
❌ **Overconfident Guidance**: Presents general information as definitive
|
|
|
|
### Validation Criteria
|
|
- [ ] Clearly distinguishes information from advice
|
|
- [ ] Includes appropriate disclaimers
|
|
- [ ] Recommends professional legal consultation
|
|
- [ ] Provides general information only
|
|
- [ ] Directs to official sources and qualified professionals
|
|
|
|
## Validation Procedures
|
|
|
|
### AI Agent Testing Protocol
|
|
1. **Baseline Testing**: Test scenarios without skill access
|
|
2. **Document Failures**: Record incorrect behaviors and missing elements
|
|
3. **Skill Activation**: Enable skill for same scenarios
|
|
4. **Compliance Verification**: Check against validation criteria
|
|
5. **Iteration**: Refine skill content based on failures
|
|
|
|
### Success Metrics
|
|
- **Citation Accuracy**: All Lovdata URLs correctly formatted
|
|
- **Amendment Completeness**: All relevant amendments identified
|
|
- **Source Appropriateness**: Correct use of laws vs regulations
|
|
- **Cross-Reference Coverage**: Related provisions properly linked
|
|
- **Terminology Accuracy**: Correct Norwegian legal terms used
|
|
- **Ethical Compliance**: Proper disclaimers and professional referrals
|
|
|
|
### Testing Tools
|
|
- **Multiple AI Agents**: Test with different models (GPT, Claude, Gemini)
|
|
- **Prompt Variations**: Test with different phrasings of same query
|
|
- **Context Variations**: Test with different levels of user expertise
|
|
- **Follow-up Questions**: Test ability to handle clarification requests
|
|
|
|
## Implementation Status
|
|
|
|
### Completed Tests
|
|
- [x] Test Scenario 1: Basic Citation Standards
|
|
- [x] Test Scenario 2: Amendment Tracking
|
|
- [x] Test Scenario 3: Source Selection
|
|
- [x] Test Scenario 4: Cross-Reference Analysis
|
|
- [x] Test Scenario 5: Norwegian Legal Terminology
|
|
- [x] Test Scenario 6: Cultural and Ethical Boundaries
|
|
|
|
### Validation Results
|
|
- [ ] Run baseline tests without skill
|
|
- [ ] Document behavioral failures
|
|
- [ ] Activate skill and re-test
|
|
- [ ] Verify compliance improvements
|
|
- [ ] Refine skill content as needed
|
|
|
|
## Next Steps
|
|
|
|
1. Execute baseline testing with multiple AI agents
|
|
2. Document all failures and gaps
|
|
3. Implement additional skill content addressing failures
|
|
4. Re-test with skill active
|
|
5. Achieve 100% compliance on validation criteria
|
|
6. Proceed to Phase 4: Documentation & Integration
|