Skip to content

Conversation

@ullbergm
Copy link
Owner

Add Validating Admission Webhook for TTL Annotation Validation

Overview

This PR implements a validating admission webhook that validates TTL annotation format before objects are created or updated in Kubernetes. The webhook prevents invalid lease configurations from being applied to the cluster.

Architecture

Shared Webhook Design

  • Single webhook deployment validates ALL LeaseController GVKs (highly efficient)
  • Dynamic configuration: Webhook watches LeaseController CRs and automatically updates ValidatingWebhookConfiguration
  • Per-GVK control: Each LeaseController CR can independently enable/disable validation and set failure policy

Key Components

  1. Webhook Server (cmd/webhook/main.go) - Separate binary that runs as a deployment
  2. Dynamic Validator (pkg/webhook/validator.go) - Validates TTL format using existing ParseFlexibleDuration
  3. Config Manager (pkg/webhook/config_manager.go) - Watches LeaseController CRs and manages webhook rules
  4. Finalizer-based Cleanup - Ensures proper removal of webhook configuration on CR deletion

Features

Validates TTL annotation format (2d, 1h30m, 1w, etc.)
Configurable per-GVK - Enable/disable validation for each resource type
Configurable failure policy - Ignore (soft fail) or Fail (hard fail) per GVK
Automatic certificate management - Uses cert-manager for TLS certificates
High availability - 2 replicas with proper health checks
Security hardened - Non-root, no capabilities, seccomp profile
Finalizer-based cleanup - Guaranteed removal of webhook config on deletion
100% backward compatible - Disabled by default, no breaking changes

Configuration Example

apiVersion: object-lease-controller.ullberg.io/v1
kind: LeaseController
metadata:
  name: leasecontroller-deployments
spec:
  group: "apps"
  version: "v1"
  kind:
    singular: "Deployment"
    plural: "deployments"
  webhook:
    enabled: true            # Enable validation for this GVK
    failurePolicy: Fail      # Reject invalid objects (or "Ignore")

How It Works

User creates object with TTL annotation
↓
Kubernetes API Server
↓
ValidatingWebhookConfiguration (checks if GVK matches rules)
↓
Webhook Service → Webhook Pod
↓
ConfigManager.ShouldValidate(GVK) → true/false
↓
Validator.validate(TTL annotation)
↓
util.ParseFlexibleDuration(TTL)
↓
Response: Allowed=true/false + error message

Cleanup & Finalizers

The webhook uses Kubernetes finalizers to ensure proper cleanup:

  1. Finalizer added: webhook.object-lease-controller.ullberg.io/finalizer when webhook is enabled
  2. Deletion handling: When LeaseController is deleted:
    • Removes GVK from ValidatingWebhookConfiguration
    • Deletes ValidatingWebhookConfiguration if no rules remain
    • Removes finalizer to allow CR deletion
  3. Guaranteed cleanup: Prevents orphaned webhook rules even if CR is force-deleted

Testing

All existing tests pass (100% coverage for controllers/metrics/util)
Webhook binary compiles successfully
Main controller binary compiles successfully
golangci-lint: 0 issues
No breaking changes to existing functionality

Security

  • Non-root execution (UID 65532)
  • Minimal capabilities (drops ALL)
  • Read-only filesystem
  • Seccomp profile enabled
  • TLS 1.2+ encryption for all webhook calls
  • Automatic certificate rotation via cert-manager

Performance

  • Shared architecture: One deployment for all GVKs (efficient)
  • Selective validation: Only validates objects with TTL annotation
  • In-memory caching: GVK configurations cached in memory
  • High availability: 2 replicas with proper resource limits

Dependencies

Required

  • cert-manager: For automatic TLS certificate management
  • Kubernetes 1.16+: For admissionregistration.k8s.io/v1

Optional

  • None (webhook is entirely optional feature)

Backward Compatibility

Fully backward compatible:

  • Webhook disabled by default (webhook.enabled: false)
  • Existing LeaseController CRs continue working without changes
  • No breaking changes to existing functionality
  • Webhook deployment is separate from controller deployments

Documentation

  • docs/webhook.md: Comprehensive webhook documentation

    • Architecture overview with diagram
    • Configuration examples
    • Certificate management guide
    • Troubleshooting guide
    • Security considerations
  • README.md: Updated with webhook feature in features list

Future Enhancements

  • Add unit tests for webhook validation logic
  • Add integration tests with real Kubernetes API server
  • Add Prometheus metrics for validation requests/failures
  • Support for custom CA bundles
  • Namespace filtering options

Closes

Addresses the need for early validation of TTL annotations, preventing invalid configurations from being applied to objects.

@github-actions github-actions bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 18, 2025
@ullbergm ullbergm removed the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 18, 2025
@github-actions github-actions bot added the area/docs Changes made to the documentation. label Nov 18, 2025
@ullbergm ullbergm added the new-feature New features or options. label Nov 18, 2025
@github-actions github-actions bot added the area/source Changes to the source. label Nov 18, 2025
@codecov
Copy link

codecov bot commented Nov 18, 2025

Codecov Report

❌ Patch coverage is 65.56604% with 146 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
cmd/webhook/main.go 6.79% 96 Missing ⚠️
pkg/webhook/config_manager.go 82.07% 35 Missing and 3 partials ⚠️
pkg/webhook/validator.go 88.99% 10 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

@github-actions github-actions bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. area/github Changes made in the github directory. labels Nov 19, 2025
@ullbergm ullbergm force-pushed the add-validating-webhook branch 2 times, most recently from b7f6752 to b924dbe Compare November 26, 2025 19:33
Signed-off-by: Magnus Ullberg <magnus@ullberg.us>
@ullbergm ullbergm force-pushed the add-validating-webhook branch from b924dbe to 423ebce Compare November 26, 2025 19:34
Signed-off-by: Magnus Ullberg <magnus@ullberg.us>
@ullbergm ullbergm closed this Nov 28, 2025
@github-actions
Copy link

github-actions bot commented Dec 6, 2025

Pull Request closed and locked due to lack of activity.
If you'd like to build on this closed PR, you can clone it using this method: https://stackoverflow.com/a/14969986
Then open a new PR, referencing this closed PR in your message.

@github-actions github-actions bot locked and limited conversation to collaborators Dec 6, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area/docs Changes made to the documentation. area/github Changes made in the github directory. area/source Changes to the source. new-feature New features or options. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants