Major MCP Server Cleanup + UI Fixes - Sprint 6

COMPLETED WORK:
===============

1. MCP Server Complete Cleanup:
   - Created mcp_tool.py decorator with auto parameter extraction
   - Removed 460 lines of hardcoded tool definitions
   - Dynamic tool loading from modules (22 tools working)
   - Clean architecture: 350 lines vs 1000+ before

2. UI-MCP State Representation Fix:
   - Fixed critical bug: parameters not showing in UI
   - UI now directly represents MCP server state (no mapping)
   - Native flat schema support
   - Added design principle documentation in US-RAG-001

3. Tools Working (22 total):
   - File Access Tools (6): read, write, list, search, info, exists
   - Database Tools (6): query, schema, timeline, statistics
   - RAG Swarm Tools (4): query, search, analyze, stats
   - Google Drive Tools (6): list, search, get, upload, create, status

4. Documentation Updates:
   - Updated US-RAG-001 with design principles
   - Added pending work items for next session
   - Cleaned up agile directory structure
   - Updated sprint organization standards

PENDING FOR NEXT SESSION:
=========================
1. Google Docs MCP tools (similar to Drive)
2. API Key Configuration UI
3. Fix Research Swarm tools (import error)
4. Fix Agile tools (import error)
5. Fix Link Integrity tools (missing decorators)

FILES CHANGED:
==============
- utils/mcp/mcp_tool.py (NEW - 200 lines)
- utils/mcp/server.py (REWRITE - 350 lines, was 1000+)
- apps/mcp_management_app.py (UI parameter fix)
- docs/agile/sprints/sprint_6/user_stories/US-RAG-001.md (updated)

Author: gitwalter

.cursor/rules/context/agile_coordination.mdc

@@ -81,6 +81,58 @@ def agile_coordination_system(context: AgileContext) -> CoordinationResult:
     return coordination
 ```
 
+## Sprint Directory Organization
+
+### **CRITICAL: Numbered Sprint Structure**
+```yaml
+# MANDATORY: All sprint work uses numbered directories
+sprint_structure:
+  canonical_directories:
+    - "docs/agile/sprints/sprint_0/"  # Sprint 0
+    - "docs/agile/sprints/sprint_1/"  # Sprint 1
+    - "docs/agile/sprints/sprint_N/"  # Future sprints
+  
+  pointer_file:
+    location: "docs/agile/sprints/current_sprint.md"
+    purpose: "Quick reference dashboard pointing to active sprint"
+    content: "High-level status, links to full sprint directory"
+  
+  # FORBIDDEN: Do not create these directories
+  forbidden:
+    - "docs/agile/sprints/current/"        # Use numbered sprints
+    - "docs/agile/sprints/sprint_current/" # Use numbered sprints
+    - "docs/agile/sprints/active/"         # Use current_sprint.md file
+
+# Sprint Directory Contents (for each sprint_N/)
+sprint_directory_structure:
+  required_files:
+    - "README.md"           # Sprint overview
+    - "backlog.md"          # Sprint backlog
+    - "planning.md"         # Planning documents
+    - "review.md"           # Sprint review
+    - "retrospective.md"    # Retrospective
+  
+  required_directories:
+    - "user_stories/"       # All user stories for this sprint
+    - "analysis/"           # Analysis documents
+    - "planning/"           # Detailed planning docs
+    - "completion_summaries/" # Story completion summaries
+  
+  optional_directories:
+    - "daily_standups/"     # Daily standup notes
+    - "metrics/"            # Sprint metrics
+    - "blockers/"           # Blocker tracking
+    - "retrospectives/"     # Detailed retrospectives
+
+# Usage Pattern
+workflow:
+  1_check_active_sprint: "Read docs/agile/sprints/current_sprint.md"
+  2_work_in_numbered_sprint: "All work goes in sprint_N/ directory"
+  3_update_pointer: "Update current_sprint.md with progress"
+  4_close_sprint: "Complete all artifacts in sprint_N/"
+  5_start_next_sprint: "Create sprint_N+1/ directory"
+```
+
 ## 1. Systematic Artifact Management
 
 ### **Live Artifact System**
@@ -92,6 +144,7 @@ class AgileArtifactSystem:
     def __init__(self):
         self.artifact_registry = self._load_artifact_registry()
         self.automation_engine = ArtifactAutomationEngine()
+        self.sprint_organizer = SprintDirectoryOrganizer()
 
     def maintain_artifacts(self, change_context: ChangeContext) -> ArtifactMaintenance:
         """Automatically maintain artifacts based on changes."""
@@ -123,6 +176,14 @@ class AgileArtifactSystem:
         if context.involves_user_stories:
             affected.append(self.artifact_registry.user_story_catalog)
             affected.append(self.artifact_registry.sprint_summary)
+            
+            # CRITICAL: Ensure user story is in correct sprint directory
+            sprint_num = self._get_active_sprint_number()
+            user_story_path = f"docs/agile/sprints/sprint_{sprint_num}/user_stories/"
+            self.sprint_organizer.ensure_story_in_sprint_directory(
+                context.user_story_id, 
+                user_story_path
+            )
 
         # Code changes affect epic overview
         if context.involves_code_changes:
@@ -131,23 +192,38 @@ class AgileArtifactSystem:
         # Sprint changes affect all sprint artifacts
         if context.involves_sprint_activities:
             affected.extend(self.artifact_registry.sprint_artifacts)
+            
+            # Update current_sprint.md pointer
+            self.sprint_organizer.update_current_sprint_pointer(context)
 
         return affected
+    
+    def _get_active_sprint_number(self) -> int:
+        """Get active sprint number from current_sprint.md."""
+        current_sprint_file = Path("docs/agile/sprints/current_sprint.md")
+        if current_sprint_file.exists():
+            content = current_sprint_file.read_text()
+            # Extract sprint number from "Sprint Number: 6" format
+            import re
+            match = re.search(r'Sprint Number[:\s]+(\d+)', content)
+            if match:
+                return int(match.group(1))
+        return 0  # Default to sprint 0 if not found
 ```
 
 ### **Artifact Types and Automation**
 ```yaml
 # Core Agile Artifacts
 EPIC_OVERVIEW:
-  location: "docs/agile/EPIC_OVERVIEW.md"
+  location: "docs/agile/catalogs/epic-overview.md"
   automation: "Update on epic completion or modification"
 
 USER_STORY_CATALOG:
-  location: "docs/agile/USER_STORY_CATALOG.md"
+  location: "docs/agile/catalogs/USER_STORY_CATALOG.md"
   automation: "Update on any user story status change"
 
 SPRINT_SUMMARY:
-  location: "docs/agile/SPRINT_SUMMARY.md"
+  location: "docs/agile/catalogs/SPRINT_SUMMARY.md"
   automation: "Update daily with sprint progress"
 
 SPRINT_ARTIFACTS:

.cursor/rules/context/unified_test_developer_agent_rule.mdc

@@ -1,3 +1,35 @@
+---
+alwaysApply: false
+autoFix: true
+category: context-testing
+contexts:
+- TESTING
+- TEST_DEVELOPMENT
+- DEBUGGING
+dependencies:
+- systematic_completion
+- development_excellence
+description: Unified Test-Developer Agent - Auto-activate for test failures and systematic fixing
+enforcement: warning
+formalLayer: context
+globs:
+- '**/*'
+languageGames:
+- test_driven_development
+- systematic_debugging
+- boy_scout_improvement
+linguisticFramework: testing
+logicalType: 2
+priority: high
+tags:
+- context
+- testing
+- unified_agent
+- type_2
+- priority_3
+tier: '2'
+---
+
 # Unified Test-Developer Agent Rule
 
 **CRITICAL**: Automatically activate unified test-developer agent mode when encountering test failures or development tasks that require systematic testing and fixing. This rule combines Boy Scout principles, test-driven development, courage for completion, and systematic problem-solving into one seamless auto-mode operation.

.cursor/rules/core/file_organization_enforcement.mdc

@@ -1,11 +1,30 @@
 ---
-description: "Active file organization enforcement - prevents misplaced files"
-category: "core"
-priority: "critical"
 alwaysApply: true
-globs: ["**/*"]
-tags: ['file-organization', 'enforcement', 'active']
-tier: "1"
+autoFix: true
+category: core-organization
+contexts:
+- ALL
+dependencies:
+- safety_first_principle
+- systematic_completion
+description: Active file organization enforcement - prevents misplaced files
+enforcement: error
+formalLayer: core
+globs:
+- '**/*'
+languageGames:
+- file_organization
+- project_structure
+linguisticFramework: foundational
+logicalType: 1
+priority: critical
+tags:
+- core
+- file-organization
+- enforcement
+- active
+- type_1
+tier: '1'
 ---
 
 # File Organization Enforcement Rule

.cursor/rules/core/systematic_completion.mdc

@@ -537,6 +537,46 @@ def handle_user_request(request):
 - **Technical Docs**: Update architecture/design docs when they become stale
 - **Bug Fixes**: Document in commit messages, not separate files
 
+### **TODO List Discipline**
+
+**CRITICAL**: ALWAYS maintain an active TODO list during work sessions:
+
+```python
+# REQUIRED: Start every work session by creating/updating TODO list
+def start_work_session():
+    todo_write(todos=[
+        {"id": "task-1", "content": "Description", "status": "in_progress"},
+        {"id": "task-2", "content": "Description", "status": "pending"}
+    ])
+    
+# REQUIRED: Update TODO list as work progresses
+def complete_task():
+    complete_implementation()
+    todo_write(todos=[
+        {"id": "task-1", "status": "completed"}
+    ], merge=True)
+    
+# FORBIDDEN: Working without an active TODO list
+def work_without_tracking():
+    # ❌ No TODO list = No systematic completion
+    do_work_blindly()  # FORBIDDEN
+```
+
+**TODO List Requirements**:
+- ✅ **ALWAYS create** TODO list for multi-step tasks (3+ steps)
+- ✅ **ALWAYS update** TODO status as tasks complete
+- ✅ **ALWAYS mark** current task as "in_progress"
+- ✅ **ALWAYS complete** all TODOs before ending session
+- ❌ **NEVER work** without active TODO list on complex tasks
+- ❌ **NEVER leave** unfinished TODOs without user input
+
+**Benefits of TODO Lists**:
+- ✅ Systematic tracking of progress
+- ✅ Clear visibility of remaining work
+- ✅ Prevents forgotten tasks
+- ✅ Demonstrates thoroughness
+- ✅ User can see progress at any time
+
 ## Remember
 
 **"Always leave things better than you found them."**

.cursor/rules/enforcement/AUTOMATION_SCRIPT_ENFORCEMENT_RULE.mdc

@@ -152,6 +152,32 @@ setup_automated_git_pull.py:
   mandatory_for: ["repository synchronization", "automatic updates"]
 ```
 
+### **File Operations & Link Integrity** (MANDATORY USAGE)
+```yaml
+link_healing_system.py:
+  location: "scripts/link_healing_system.py"
+  mcp_tools: "utils/mcp/tools/link_integrity_tools.py"
+  triggers: ["file rename", "file move", "directory reorganization", "link validation"]
+  mandatory_for: ["any file operations that could break documentation links"]
+  description: "Prevents broken links during file operations"
+  
+  # MCP Tool Integration (5 tools available)
+  mcp_tool_ids:
+    - "link.scan_all" (scan project for all links)
+    - "link.validate" (validate and find broken links)
+    - "link.heal" (heal links after file renames)
+    - "link.generate_report" (generate link analysis report)
+    - "link.check_before_move" (check impact before moving files)
+    
+  # Usage Pattern
+  workflow:
+    1_before_rename: "python scripts/link_healing_system.py  # Scan and validate"
+    2_check_impact: "Use link.check_before_move MCP tool"
+    3_perform_rename: "Rename/move files"
+    4_heal_links: "Use link.heal MCP tool with rename mapping"
+    5_verify: "Use link.validate to confirm no broken links"
+```
+
 ## 2. Enforcement Mechanisms
 
 ### **Operation Interception**
@@ -207,6 +233,12 @@ class ContextAutomationEnforcer:
         "BUILD_DEPLOY": {
             "daily_builds": ["daily_build_automation.py"],
             "setup_validation": ["setup_status_automation.py"]
+        },
+        "FILE_OPERATIONS": {
+            "file_rename": ["link_healing_system.py"],
+            "file_move": ["link_healing_system.py"],
+            "directory_reorganization": ["link_healing_system.py"],
+            "link_validation": ["link_healing_system.py"]
         }
     }
 
@@ -273,6 +305,29 @@ def enforce_git_automation(operation):
         )
 ```
 
+### **File Operations & Link Healing** (ZERO TOLERANCE)
+```python
+# FORBIDDEN: File renames/moves without link healing
+def manual_file_rename(old_path: str, new_path: str):
+    os.rename(old_path, new_path)  # BLOCKED - use link_healing_system.py
+
+def manual_directory_move(old_dir: str, new_dir: str):
+    shutil.move(old_dir, new_dir)  # BLOCKED - breaks documentation links
+
+# REQUIRED: Link healing enforcement
+def enforce_link_healing(operation):
+    if operation.type in ["file_rename", "file_move", "directory_reorganization"]:
+        # MANDATORY: Check link impact first
+        impact = check_link_impact(operation.old_path, operation.new_path)
+        
+        if impact.links_affected > 0:
+            # MANDATORY: Use link healing system
+            raise AutomationRequiredError(
+                f"LINK HEALING REQUIRED: {impact.links_affected} links affected. "
+                f"Use: python scripts/link_healing_system.py"
+            )
+```
+
 ## 4. Script Execution Framework
 
 ### **Automatic Script Discovery and Execution**

.cursor/rules/meta/deductive_inductive_rule_system_framework.mdc

@@ -6,7 +6,7 @@ contexts:
 - ALL
 dependencies: []
 description: Deductive-Inductive Rule System Framework - Meta-governance for rule hierarchy and application
-enforcement: foundational
+enforcement: blocking
 formalLayer: meta
 globs:
 - '**/*'