feat: Add comprehensive configuration validation with schema

[sapientpants]

Feature: Configuration Validation with Schema

Business Value

Prevent runtime failures and improve operational reliability by validating all configuration at startup using a schema-based approach. This feature will catch misconfigurations immediately with clear, actionable error messages, reduce debugging time from configuration issues, ensure consistent behavior across deployments, and provide confidence that the server is properly configured before accepting requests. For production deployments, this is critical for preventing silent failures and ensuring compliance with operational requirements.

User Story

As a DevOps engineer deploying the DeepSource MCP server
I want the server to validate all configuration at startup with clear error messages
So that I can quickly identify and fix any configuration issues before the server starts serving requests

Gherkin Specification

Feature: Configuration Validation with Schema
  Implement comprehensive configuration validation using Zod schema at startup,
  with required keys, type validation, range checks, and clear error messages.

  Background:
    Given the DeepSource MCP server configuration system
    And various environment variables and configuration sources
    And the need for fail-fast behavior on misconfiguration

  Scenario: Required API key validation
    Given the server is starting
    When DEEPSOURCE_API_KEY is not set
    Then startup should fail immediately
    And the error message should be:
      """
      Configuration Error: Missing required DEEPSOURCE_API_KEY
      
      The DeepSource API key is required but not configured.
      
      To fix this:
      1. Get your API key from: https://deepsource.io/settings/api-keys
      2. Set the environment variable:
         export DEEPSOURCE_API_KEY="your-api-key-here"
      
      See documentation: https://github.com/sapientpants/deepsource-mcp-server#configuration
      """
    And exit code should be 1

  Scenario: Valid configuration with defaults
    Given minimal valid configuration:
      | Variable              | Value                    |
      | DEEPSOURCE_API_KEY    | ds_abc123def456         |
    When the server starts
    Then configuration should be validated successfully
    And defaults should be applied:
      | Setting               | Default Value            |
      | API_BASE_URL          | https://api.deepsource.io/graphql/ |
      | REQUEST_TIMEOUT       | 30000                    |
      | LOG_LEVEL             | INFO (production)        |
      | CACHE_ENABLED         | true                     |
      | CACHE_TTL             | 300                      |
      | MAX_RETRIES           | 3                        |

  Scenario Outline: Type validation for configuration values
    Given configuration value <variable> = "<value>"
    When the configuration is validated
    Then validation should <result>
    And show error "<error_message>"

    Examples:
      | variable                    | value      | result | error_message                                        |
      | DEEPSOURCE_REQUEST_TIMEOUT  | abc        | fail   | REQUEST_TIMEOUT must be a positive integer          |
      | DEEPSOURCE_REQUEST_TIMEOUT  | -5000      | fail   | REQUEST_TIMEOUT must be positive (minimum: 1000ms)  |
      | DEEPSOURCE_REQUEST_TIMEOUT  | 0          | fail   | REQUEST_TIMEOUT must be positive (minimum: 1000ms)  |
      | DEEPSOURCE_REQUEST_TIMEOUT  | 30000      | pass   |                                                      |
      | LOG_LEVEL                   | INVALID    | fail   | LOG_LEVEL must be one of: DEBUG, INFO, WARN, ERROR  |
      | LOG_LEVEL                   | INFO       | pass   |                                                      |
      | CACHE_TTL                   | 3.14       | fail   | CACHE_TTL must be an integer                        |
      | MAX_RETRIES                 | 100        | fail   | MAX_RETRIES must be between 0 and 10                |

  Scenario: Range validation for numeric values
    Given various numeric configuration values
    When the configuration is validated
    Then ranges should be enforced:
      | Variable                    | Min    | Max      | Default | Unit        |
      | DEEPSOURCE_REQUEST_TIMEOUT  | 1000   | 300000   | 30000   | ms          |
      | CACHE_TTL                   | 0      | 86400    | 300     | seconds     |
      | CACHE_MAX_ENTRIES           | 10     | 10000    | 1000    | entries     |
      | CACHE_MAX_MEMORY_MB         | 1      | 1000     | 100     | MB          |
      | MAX_RETRIES                 | 0      | 10       | 3       | attempts    |
      | RETRY_BASE_DELAY            | 100    | 10000    | 1000    | ms          |
      | PAGE_SIZE                   | 1      | 100      | 50      | items       |

  Scenario: URL validation
    Given configuration for API_BASE_URL
    When the URL is validated
    Then validation should check:
      | Check                   | Valid Example                         | Invalid Example            |
      | Valid URL format        | https://api.deepsource.io/graphql/   | not-a-url                 |
      | HTTPS protocol          | https://api.example.com              | http://api.example.com    |
      | Trailing slash          | https://api.example.com/             | (adds if missing)         |
      | No auth in URL          | https://api.example.com              | https://user:pass@api.com |

  Scenario: API key format validation
    Given an API key value
    When validated
    Then it should check:
      | Format Check            | Valid                                 | Invalid                   |
      | Minimum length          | ds_abc123def456ghij789                | ds_123                    |
      | Maximum length          | (up to 256 chars)                    | (over 256 chars)          |
      | No whitespace           | ds_abc123def456                      | "ds_abc 123"              |
      | No common placeholders  | ds_real_key_value                    | your-api-key-here         |

  Scenario: Configuration schema using Zod
    Given a Zod schema for configuration
    When the server starts
    Then the schema should validate:
      ```typescript
      const ConfigSchema = z.object({
        // Required
        apiKey: z.string()
          .min(10, "API key too short")
          .max(256, "API key too long")
          .regex(/^[^\s]+$/, "API key cannot contain whitespace")
          .refine(val => !val.includes('your-api-key'), "Replace placeholder API key"),
        
        // URLs
        apiBaseUrl: z.string()
          .url("Invalid URL format")
          .startsWith("https://", "Must use HTTPS")
          .transform(url => url.endsWith('/') ? url : url + '/'),
        
        // Timeouts and delays
        requestTimeout: z.number()
          .int("Must be an integer")
          .min(1000, "Minimum 1 second")
          .max(300000, "Maximum 5 minutes")
          .default(30000),
        
        // Logging
        logLevel: z.enum(['DEBUG', 'INFO', 'WARN', 'ERROR'])
          .default(process.env.NODE_ENV === 'production' ? 'INFO' : 'DEBUG'),
        
        // Cache configuration
        cache: z.object({
          enabled: z.boolean().default(true),
          ttl: z.number().int().min(0).max(86400).default(300),
          maxEntries: z.number().int().min(10).max(10000).default(1000),
          maxMemoryMB: z.number().int().min(1).max(1000).default(100),
        }),
        
        // Retry configuration
        retry: z.object({
          maxAttempts: z.number().int().min(0).max(10).default(3),
          baseDelay: z.number().int().min(100).max(10000).default(1000),
          maxDelay: z.number().int().min(1000).max(60000).default(30000),
        }),
        
        // Feature flags
        features: z.object({
          serverSideFiltering: z.boolean().default(true),
          caching: z.boolean().default(true),
          retries: z.boolean().default(true),
          correlationIds: z.boolean().default(true),
        }),
      });
      ```

  Scenario: Multiple validation errors
    Given multiple configuration errors:
      | Variable                    | Value     | Issue                       |
      | DEEPSOURCE_API_KEY          | (missing) | Required field              |
      | DEEPSOURCE_REQUEST_TIMEOUT  | abc       | Not a number                |
      | LOG_LEVEL                   | TRACE     | Invalid enum value          |
    When validation runs
    Then all errors should be reported together:
      """
      Configuration Validation Failed (3 errors):
      
      1. DEEPSOURCE_API_KEY: Required field is missing
         Fix: Set environment variable DEEPSOURCE_API_KEY
      
      2. DEEPSOURCE_REQUEST_TIMEOUT: Expected number, received string "abc"
         Fix: Set to a number between 1000 and 300000
      
      3. LOG_LEVEL: Invalid enum value "TRACE"
         Fix: Use one of: DEBUG, INFO, WARN, ERROR
      
      Configuration documentation: https://github.com/sapientpants/deepsource-mcp-server#configuration
      """

  Scenario: Environment-specific validation
    Given the NODE_ENV environment variable
    When set to <environment>
    Then validation should enforce <rules>

    Examples:
      | environment | rules                                                    |
      | production  | Require HTTPS URLs, INFO log level minimum              |
      | development | Allow HTTP URLs, any log level                          |
      | test        | Use test defaults, skip external service checks         |

  Scenario: Configuration file support
    Given a configuration file at .deepsource-mcp.json
    When the server starts
    Then configuration should be loaded in order:
      | Priority | Source                    | Override Behavior           |
      | 1        | Default values            | Base configuration          |
      | 2        | Config file               | Overrides defaults          |
      | 3        | Environment variables     | Overrides file and defaults |
    And validate the merged configuration

  Scenario: Helpful error messages
    Given a configuration error
    When displayed to the user
    Then the message should include:
      | Component             | Example                                           |
      | Error summary         | "Configuration Error: Invalid REQUEST_TIMEOUT"    |
      | Current value         | "Current value: 'abc'"                           |
      | Expected format       | "Expected: positive integer (milliseconds)"       |
      | Valid range           | "Valid range: 1000-300000"                       |
      | How to fix            | "Set DEEPSOURCE_REQUEST_TIMEOUT=30000"           |
      | Documentation link    | "See: https://docs.../configuration#timeouts"    |

  Scenario: Configuration reporting
    Given a valid configuration
    When the server starts with --show-config flag
    Then it should display:
      """
      DeepSource MCP Server Configuration:
      =====================================
      API:
        Base URL: https://api.deepsource.io/graphql/
        Timeout: 30000ms
        API Key: ds_****6789 (masked)
      
      Logging:
        Level: INFO
        File: /var/log/deepsource-mcp.log
      
      Cache:
        Enabled: true
        TTL: 300 seconds
        Max Entries: 1000
        Max Memory: 100 MB
      
      Retry:
        Max Attempts: 3
        Base Delay: 1000ms
        Max Delay: 30000ms
      
      Features:
        Server-side Filtering: enabled
        Caching: enabled
        Retries: enabled
        Correlation IDs: enabled
      
      Configuration validated successfully ✓
      """

Acceptance Criteria

• Zod schema implementation for all configuration:
  • Required fields validation
  • Type checking for all values
  • Range validation for numeric values
  • Format validation for URLs and API keys
  • Enum validation for categorical values
• Fail-fast behavior:
  • Exit immediately on missing DEEPSOURCE_API_KEY
  • Exit code 1 on any validation failure
  • No server startup on invalid configuration
• Clear error messages:
  • Specific field that failed
  • Current invalid value
  • Expected format/range
  • How to fix the issue
  • Documentation links
• Sensible defaults:
  • All optional fields have defaults
  • Environment-aware defaults (dev vs prod)
  • Documented default values
• Multiple error reporting:
  • Collect all validation errors
  • Display all at once
  • Numbered list format
• Configuration sources:
  • Environment variables (highest priority)
  • Configuration file (optional)
  • Built-in defaults
• Security:
  • Mask sensitive values in logs
  • Validate API key format
  • Enforce HTTPS in production
• Developer experience:
  • --show-config flag for debugging
  • --validate-config for CI/CD
  • Example configuration file

Non-Goals

• Will NOT implement dynamic configuration reloading
• Will NOT add configuration UI/dashboard
• Will NOT store configuration in database
• Will NOT implement configuration encryption at rest
• Out of scope: Multi-tenant configuration
• Will NOT add configuration versioning
• Will NOT implement remote configuration

Risks & Mitigations

• Risk: Breaking existing deployments with stricter validation
  Mitigation: Provide migration period with warnings before hard failures

• Risk: Too strict validation preventing valid use cases
  Mitigation: Allow relaxed mode via STRICT_VALIDATION=false

• Risk: Sensitive data in error messages
  Mitigation: Always mask sensitive values in output

• Risk: Complex configuration overwhelming users
  Mitigation: Provide minimal required config with smart defaults

Technical Considerations

• Architecture impact:

  • Add Zod as dependency
  • Refactor config module to use schema
  • Add validation at startup
  • Implement config file loading

• Implementation with existing config module:

  // src/config/schema.ts
  import { z } from 'zod';
  
  export const ConfigSchema = z.object({
    apiKey: z.string().min(10).refine(/* validation */),
    apiBaseUrl: z.string().url().startsWith('https://'),
    // ... rest of schema
  });
  
  // src/config/index.ts
  import { ConfigSchema } from './schema';
  
  export function validateConfig(config: unknown): DeepSourceConfig {
    try {
      return ConfigSchema.parse(config);
    } catch (error) {
      if (error instanceof z.ZodError) {
        throw formatValidationError(error);
      }
      throw error;
    }
  }

• Error formatting:

  function formatValidationError(error: z.ZodError): Error {
    const issues = error.issues.map((issue, i) => {
      return `${i + 1}. ${issue.path.join('.')}: ${issue.message}
         Fix: ${getSuggestion(issue)}`;
    });
    
    return new Error(`
      Configuration Validation Failed (${error.issues.length} errors):
      
      ${issues.join('\n\n')}
      
      See: https://docs.../configuration
    `);
  }

Testing Requirements

• Unit tests for schema validation
• Integration tests for config loading
• Tests for each validation rule
• Error message format tests
• Default value tests
• Environment-specific tests
• Config file loading tests
• Priority/override tests
• Security masking tests

Definition of Done

• Zod schema covers all configuration
• Validation runs at startup
• Missing API key fails fast
• All validation rules implemented
• Clear error messages with fixes
• Defaults applied correctly
• Config file support added
• --show-config flag working
• Documentation updated
• Migration guide created
• All tests passing >95% coverage
• No breaking changes for valid configs
• Performance impact <10ms
• Code reviewed and approved

Implementation Notes

1. Phased Implementation:

   • Phase 1: Add Zod and create schema
   • Phase 2: Integrate with existing config module
   • Phase 3: Add config file support
   • Phase 4: Improve error messages
   • Phase 5: Add CLI flags

2. Configuration Hierarchy:

   1. Built-in defaults (lowest priority)
   2. Config file (.deepsource-mcp.json)
   3. Environment variables (highest priority)
   

3. Validation Categories:

   • Critical: API key, base URL
   • Important: Timeouts, retries
   • Optional: Cache, features
   • Development: Debug settings

4. Error Message Template:

   [FIELD]: [WHAT'S WRONG]
   Current: [CURRENT_VALUE]
   Expected: [EXPECTED_FORMAT]
   Fix: [HOW_TO_FIX]
   Docs: [LINK]
   

5. Testing Strategy:

   • Test valid configurations
   • Test each validation failure
   • Test combined errors
   • Test default application
   • Test environment overrides

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com