LinkML Rust Implementation

A high-performance, production-ready Rust implementation of LinkML with 100% feature parity with the Python reference implementation. This library provides schema validation, code generation, and TypeQL generation capabilities with exceptional performance and safety guarantees.

Features

• ✅ 100% Python LinkML Parity - Complete feature compatibility with Python LinkML
• 🚀 High Performance - 126x faster TypeQL generation, 10x faster validation
• 🛡️ Production Ready - Comprehensive security audit, 500+ tests
• 🔧 Multi-Language Code Generation - Generate code for 10+ target languages
• 📊 Advanced Validation - Rules engine, boolean constraints, conditional requirements
• 🎯 TypeQL Generator - Exceptional performance (0.79ms for 100 classes)
• 💾 Batch Processing - Handle 100k+ records/second
• 🔒 Secure - Expression sandboxing, resource limits, injection protection
• 📦 Modular Architecture - Clean separation of concerns with core, service, and client crates

Quick Start

Installation

Add to your Cargo.toml:

[dependencies]
linkml-service = "2.0.0"
linkml-core = "2.0.0"
tokio = { version = "1.43", features = ["full"] }
serde_json = "1.0"

Basic Usage

use linkml_service::{create_linkml_service, LinkMLService};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create the LinkML service
    let linkml = create_linkml_service().await?;

    // Load and validate a schema
    let schema = linkml.load_schema("person_schema.yaml").await?;

    // Validate data against the schema
    let data = serde_json::json!({
        "name": "John Doe",
        "email": "john@example.com",
        "age": 30
    });

    let result = linkml.validate_data(&schema, &data, "Person").await?;

    if result.is_valid() {
        println!("✅ Data is valid!");
    } else {
        println!("❌ Validation errors: {:?}", result.errors());
    }

    Ok(())
}

Generate TypeQL

use linkml_service::{create_linkml_service, LinkMLService};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let linkml = create_linkml_service().await?;

    // Load schema
    let schema = linkml.load_schema("schema.yaml").await?;

    // Generate TypeQL
    let typeql = linkml.generate_typeql(&schema).await?;

    // Save to file
    std::fs::write("schema.tql", typeql)?;

    println!("✅ TypeQL schema generated!");

    Ok(())
}

Project Structure

This project is organized into multiple crates for modularity and clarity:

linkml/
├── core/              # Core types, traits, and error definitions
├── service/           # Main LinkML validation and code generation service
├── client/            # Client library for interacting with LinkML services
├── build-tools/       # Build-time tools and cargo plugins
├── ide-plugins/       # IDE integration plugins
├── scripts/           # Utility scripts
├── schemas/           # Example LinkML schemas
└── docs/              # Comprehensive documentation

RootReal LinkML Schemata Location:
crates/model/symbolic/schemata/  # Production LinkML schemas for RootReal
├── meta/              # Meta-level schemas (identifiers, labels, descriptions, etc.)
├── place/             # Geographic and political entities
└── ...                # Other domain-specific schemas

Crates

• linkml-core - Core types, traits, and foundational functionality
• linkml-service - Main validation and code generation service
• linkml-client - Client library for LinkML services

Production-Ready HTTP Imports

The LinkML import resolver integrates with RootReal's external API service for production-ready HTTP imports with:

• Rate Limiting - Prevents overwhelming external schema servers
• Response Caching - Avoids redundant fetches of the same schema
• Retry Logic - Handles transient network failures gracefully
• Request Logging - Tracks all external schema fetches for debugging
• Authentication - Supports private schema repositories
• Connection Pooling - Better performance for multiple imports

Using External API Client

use linkml_service::parser::{ImportResolverV2, SchemaLoader};
use external_api_service::http::client::StandardHttpClient;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create HTTP client with all production features
    let http_client = StandardHttpClient::new(
        config,
        logger,
        hash_service,
        cache_service,
        rate_limiting_service,
    )?;

    // Create import resolver with external API client
    let resolver = ImportResolverV2::with_http_client(Arc::new(http_client));

    // Create schema loader
    let loader = SchemaLoader::with_resolver(resolver);

    // Load schema with imports (uses production-ready HTTP client)
    let schema = loader.load_file("schema.yaml").await?;

    Ok(())
}

See examples/import_with_external_api.rs for a complete example.

RootReal/TextPast LinkML Conventions

RootReal uses enhanced LinkML conventions for schema and instance management:

Schema Location

• Production schemas: crates/model/symbolic/schemata/
• Legacy location: domain/schema/ (deprecated)

Schema Files

Schema files follow the pattern: {domain}/{subdomain}/schema.yaml

Example: crates/model/symbolic/schemata/place/polity/country/schema.yaml

id: https://textpast.org/schema/place/polity/country
name: country
version: 1.0.0
created_on: '2025-01-22T16:39:26+01:00'
last_updated_on: '2025-01-22T16:39:26+01:00'
imports:
  - linkml:types
  - txp:meta/entity/hyperentity/schema
  - txp:meta/label/label/schema

Instance Files

Instance files explicitly reference their schema and contain metadata:

Example: crates/model/symbolic/schemata/place/polity/country/iso_3166_entity.yaml

id: https://textpast.org/instance/place/polity/country/iso_3166_entity
schema: https://textpast.org/schema/place/polity/country/schema
name: iso_3166_entity
version: 1.0.0
created_on: "2025-03-30T10:41:26+01:00"
instances:
  - id: "US"
    label: "United States of America"
    tld: ".us"
    exact_mappings:
      - wd:Q30

Import Resolution (txp: prefix)

The txp: prefix resolves to TextPast schemas with intelligent fallback:

1. Local-first: Check crates/model/symbolic/schemata/ (uses Cache Service)
2. Remote fallback: Fetch from https://textpast.org/ if not found locally

Schema imports: txp:meta/entity/hyperentity/schema →

• Local: crates/model/symbolic/schemata/meta/entity/hyperentity/schema.yaml
• Remote: https://textpast.org/schema/meta/entity/hyperentity

Instance imports: txp:place/polity/country/iso_3166_entity →

• Local: crates/model/symbolic/schemata/place/polity/country/iso_3166_entity.yaml
• Remote: https://textpast.org/instance/place/polity/country/iso_3166_entity

Note: Schema paths end with /schema, instance paths end with the entity name.

Documentation

Comprehensive documentation is available in the docs/ directory:

• Getting Started - Quick start guide
• User Guide - Comprehensive usage documentation
• Developer Guide - For contributors and developers
• API Documentation - Complete API reference
• Architecture - System architecture and design
• Migration Guide - Migrating from Python LinkML
• Performance - Performance benchmarks and optimization
• Security - Security features and best practices

Performance

The Rust implementation offers significant performance improvements:

Operation	Python LinkML	Rust LinkML	Speedup
TypeQL Generation (100 classes)	100ms	0.79ms	126x
Validation (compiled)	10ms	1ms	10x
Batch Processing	10k/sec	100k+/sec	10x
Schema Loading	50ms	5ms	10x

See PERFORMANCE.md for detailed benchmarks.

Security

The LinkML Rust implementation includes comprehensive security features:

• Expression language sandboxing with resource limits
• Protection against ReDoS (Regular Expression Denial of Service)
• Input validation for all user data
• Secure file path handling
• No unsafe code in critical paths

See SECURITY.md for the complete security audit.

Feature Parity

This implementation achieves 100% feature parity with Python LinkML 1.x:

• ✅ Schema parsing (YAML/JSON)
• ✅ Complete validation (types, patterns, ranges, cardinality)
• ✅ Boolean constraints (exactly_one_of, any_of, all_of, none_of)
• ✅ Conditional requirements (if/then validation)
• ✅ Rules engine with complex validation logic
• ✅ Expression language with security sandboxing
• ✅ Pattern interpolation with named captures
• ✅ Unique key validation
• ✅ Code generation for 10+ languages
• ✅ TypeQL generation for TypeDB
• ✅ Schema migration with diff detection
• ✅ SchemaView for efficient schema navigation
• ✅ Compiled validators for performance

See 100_PERCENT_PARITY_ACHIEVED.md for details.

Examples

The service/examples/ directory contains comprehensive examples:

• basic_usage.rs - Basic validation and schema loading
• typeql_generation.rs - TypeQL generation examples
• code_generation.rs - Multi-language code generation
• batch_processing.rs - High-throughput batch validation
• custom_rules.rs - Custom validation rules
• migration.rs - Schema migration and diffing

Run an example:

cargo run --example basic_usage

Testing

The project includes extensive test coverage (500+ tests):

# Run all tests
cargo test --all-features

# Run unit tests only
cargo test --lib

# Run integration tests
cargo test --test integration_test

# Run with coverage
cargo tarpaulin --all-features --out Html

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Development Setup

1. Clone the repository:

   git clone https://github.com/simonckemper/rootreal.git
   cd rootreal/crates/model/symbolic/linkml

2. Install Rust (2024 edition required):

   curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

3. Build the project:

   cargo build --all-features

4. Run tests:

   cargo test --all-features

5. Check code quality:

   cargo clippy --all-targets --all-features
   cargo fmt --all -- --check

Versioning

This project follows Semantic Versioning 2.0.0. See CHANGELOG.md for release history.

License

This project is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC-BY-NC-4.0).

You are free to:

• Share — copy and redistribute the material in any medium or format
• Adapt — remix, transform, and build upon the material

Under the following terms:

• Attribution — You must give appropriate credit
• NonCommercial — You may not use the material for commercial purposes

See LICENSE for the full license text.

Authors

• Simon C. Kemper textpast@textpast.com

Acknowledgments

• The LinkML project and community
• The Rust community for excellent tools and libraries
• All contributors to this project

Links

• Repository: https://github.com/simonckemper/rootreal
• LinkML Website: https://linkml.io/
• LinkML Specification: https://linkml.io/linkml-model/docs/
• Issue Tracker: https://github.com/simonckemper/rootreal/issues
• RustDoc API: [Generated on docs.rs]

Status

Current Version: 2.0.0 (Production Ready)

This implementation is production-ready with comprehensive testing, security auditing, and performance optimization. It is actively maintained and used in production systems.

For questions, issues, or feature requests, please use the GitHub issue tracker.