Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Tests

on:
push:
pull_request:

jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
# Laravel 8–10 werkt nog op PHP 8.1
- php: 8.1
laravel: 8.*
- php: 8.1
laravel: 9.*
- php: 8.1
laravel: 10.*

# Laravel 11 vereist minimaal PHP 8.2
- php: 8.2
laravel: 11.*

# Laravel 12 werkt optimaal met PHP 8.3
- php: 8.3
laravel: 12.*

name: PHP ${{ matrix.php }} / Laravel ${{ matrix.laravel }}

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
extensions: mbstring, pdo, sqlite, bcmath, intl
coverage: none

- name: Cache Composer dependencies
uses: actions/cache@v4
with:
path: vendor
key: composer-${{ matrix.php }}-${{ matrix.laravel }}-${{ hashFiles('composer.lock') }}
restore-keys: composer-

- name: Configure Laravel version
run: composer require "illuminate/support:${{ matrix.laravel }}" "illuminate/database:${{ matrix.laravel }}" --no-update

- name: Install dependencies
run: composer update --prefer-dist --no-interaction

- name: Run PHPUnit tests
run: vendor/bin/phpunit --testdox --colors=always
89 changes: 53 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Ginkelsoft Laravel Encrypted Search Index

[![Tests](https://github.com/ginkelsoft-development/laravel-encrypted-search-index/actions/workflows/tests.yml/badge.svg)](https://github.com/ginkelsoft-development/laravel-encrypted-search-index/actions/workflows/tests.yml)
[![Latest Version on Packagist](https://img.shields.io/packagist/v/ginkelsoft/laravel-encrypted-search-index.svg?style=flat-square)](https://packagist.org/packages/ginkelsoft/laravel-encrypted-search-index)
[![Total Downloads](https://img.shields.io/packagist/dt/ginkelsoft/laravel-encrypted-search-index.svg?style=flat-square)](https://packagist.org/packages/ginkelsoft/laravel-encrypted-search-index)
[![License](https://img.shields.io/github/license/ginkelsoft-development/laravel-encrypted-search-index.svg?style=flat-square)](LICENSE.md)
[![Laravel](https://img.shields.io/badge/Laravel-8--12-brightgreen?style=flat-square&logo=laravel)](https://laravel.com)
[![PHP](https://img.shields.io/badge/PHP-8.1%20--%208.4-blue?style=flat-square&logo=php)](https://php.net)

## Overview

Modern applications that handle sensitive user data—such as healthcare, financial, or membership systems—must ensure that all personally identifiable information (PII) is properly encrypted at rest. However, standard encryption creates a practical challenge: **once data is encrypted, it can no longer be searched efficiently.**
Expand All @@ -21,35 +28,35 @@ When data is fully encrypted, you lose the ability to perform meaningful queries

This package removes that trade-off by introducing a **detached searchable index** that maps encrypted records to deterministic tokens.

---\n
---

## Key Features

* **Searchable encryption**: Enables exact and prefix-based searches over encrypted data.
* **Detached search index**: Tokens are stored separately from the main data, reducing exposure risk.
* **Deterministic hashing with peppering**: Each token is derived from normalized text combined with a secret pepper, preventing reverse-engineering.
* **No blind indexes in primary tables**: Encrypted fields remain opaqueonly hashed references are stored elsewhere.
* **High scalability**: Indexes can handle millions of records efficiently using native database indexes.
* **Laravel-native integration**: Fully compatible with Eloquent models, query scopes, and events.
* **Searchable encryption** Enables exact and prefix-based searches over encrypted data.
* **Detached search index** Tokens are stored separately from the main data, reducing exposure risk.
* **Deterministic hashing with peppering**Each token is derived from normalized text combined with a secret pepper.
* **No blind indexes in primary tables**Encrypted fields remain opaque; only hashed references are stored elsewhere.
* **High scalability** — Efficient for millions of records through database indexing.
* **Laravel-native integration** — Works directly with Eloquent models, query scopes, and model events.

---

## How It Works

Each model can declare specific fields as searchable. When the model is saved, a background process normalizes the field value, generates one or more hashed tokens, and stores them in a separate database table named `encrypted_search_index`.
Each model can declare specific fields as searchable. When the model is saved, the system normalizes the field value, generates one or more hashed tokens, and stores them in a separate table named `encrypted_search_index`.

When you search, the package hashes your input using the same process and retrieves matching model IDs from the index.

### 1. Token Generation

For each configured field:

* **Exact match token:** A SHA-256 hash of the normalized value plus a secret pepper.
* **Prefix tokens:** Multiple SHA-256 hashes representing progressive prefixes of the normalized text (e.g., `w`, `wi`, `wie`).
* **Exact match token:** A SHA-256 hash of the normalized value + secret pepper.
* **Prefix tokens:** Multiple SHA-256 hashes representing progressive prefixes of the normalized text (e.g. `w`, `wi`, `wie`).

### 2. Token Storage

All tokens are stored in `encrypted_search_index` with the following structure:
All tokens are stored in `encrypted_search_index`:

| model_type | model_id | field | type | token |
| ----------------- | -------- | ---------- | ------ | ------ |
Expand All @@ -65,20 +72,20 @@ Client::encryptedExact('last_names', 'Vermeer')->get();
Client::encryptedPrefix('first_names', 'Wie')->get();
```

These queries use database-level indexes for efficient lookups even on large datasets.
These use indexed lookups and remain performant even at scale.

---

## Security Model

| Threat | Mitigation |
| ----------------------- | --------------------------------------------------------------------------- |
| Database dump or breach | Tokens cannot be reversed to plaintext (salted and peppered SHA-256). |
| Statistical analysis | Tokens are fully detached; frequency analysis yields no useful correlation. |
| Insider access | No sensitive data in the index table; encrypted fields remain opaque. |
| Leaked `APP_KEY` | Does not affect token security; the pepper is stored separately in `.env`. |
| Threat | Mitigation |
| ----------------------- | ----------------------------------------------------------------- |
| Database dump or breach | Tokens cannot be reversed (salted + peppered SHA-256). |
| Statistical analysis | Tokens are detached; frequency analysis yields no correlation. |
| Insider access | No sensitive data in index table; encrypted fields remain opaque. |
| Leaked `APP_KEY` | Irrelevant for tokens; pepper is stored separately in `.env`. |

The system follows a **defense-in-depth** approach: encrypted data remains fully protected, while token search provides limited, controlled visibility for queries.
This design follows a **defense-in-depth** model: encrypted data stays secure, while search operations remain practical.

---

Expand All @@ -90,7 +97,7 @@ php artisan vendor:publish --tag=config
php artisan migrate
```

Update your `.env` file with a unique pepper:
Then add a unique pepper to your `.env` file:

```
SEARCH_PEPPER=your-random-secret-string
Expand All @@ -100,7 +107,7 @@ SEARCH_PEPPER=your-random-secret-string

## Configuration

`config/encrypted-search.php`
`config/encrypted-search.php`:

```php
return [
Expand Down Expand Up @@ -131,47 +138,57 @@ class Client extends Model
}
```

When a `Client` record is saved, its searchable tokens are automatically created or updated in the `encrypted_search_index` table.
When a record is saved, searchable tokens are automatically generated in `encrypted_search_index`.

### Searching

```php
// Exact match search
// Exact match
$clients = Client::encryptedExact('last_names', 'Vermeer')->get();

// Prefix match search
// Prefix match
$clients = Client::encryptedPrefix('first_names', 'Wie')->get();
```

### Rebuilding the Index

You can rebuild the entire search index using an Artisan command:
Rebuild indexes via Artisan:

```bash
php artisan encryption:index-rebuild "App\\Models\\Client"
```

This will reprocess all searchable fields for the specified model.

---

## Scalability and Performance

* **Optimized database lookups**: The `encrypted_search_index` table uses compound indexes for fast token-based lookups.
* **Chunked rebuilds**: The `index-rebuild` command supports chunked processing to handle large datasets efficiently.
* **Asynchronous rebuilds**: Can be safely run in queues or background jobs.
* **Indexed database lookups** for efficient token search.
* **Chunked rebuilds** for large datasets (`--chunk` option).
* **Queue-compatible** for asynchronous index rebuilds.

Unlike in-memory search systems, this index-based approach scales linearly with the size of your dataset and can efficiently handle millions of records.
The detached index structure scales linearly and supports millions of records efficiently.

---

## Compliance
## Framework Compatibility

| Laravel Version | PHP Version(s) Supported |
| --------------- | ------------------------ |
| 8.x | 8.0 – 8.1 |
| 9.x | 8.1 – 8.2 |
| 10.x | 8.1 – 8.3 |
| 11.x | 8.2 – 8.3 |
| 12.x | 8.3+ |

This approach aligns with major privacy and compliance frameworks:
The package is continuously tested across all supported combinations using GitHub Actions.

---

## Compliance

* GDPR: Minimal data exposure; encrypted and hashed data separation.
* HIPAA: Ensures ePHI remains protected even in breach scenarios.
* ISO 27001: Supports layered security controls for data confidentiality.
* **GDPR** — Encrypted and hashed separation ensures minimal data exposure.
* **HIPAA** — Meets encryption-at-rest requirements for ePHI.
* **ISO 27001** — Aligns with confidentiality and cryptographic control standards.

---

Expand Down
40 changes: 24 additions & 16 deletions composer.json
Original file line number Diff line number Diff line change
@@ -1,33 +1,41 @@
{
"name": "ginkelsoft/laravel-encrypted-search-index",
"description": "Searchable indexes for encrypted model fields in Laravel.",
"description": "Encrypted and privacy-preserving search indexing for Laravel models.",
"type": "library",
"license": "MIT",
"authors": [
{
"name": "Wietse van Ginkel",
"email": "info@ginkelsoft.com"
}
],
"require": {
"php": "^7.4 || ^8.0 || ^8.1 || ^8.2 || ^8.3",
"illuminate/support": "^8.0 || ^9.0 || ^10.0 || ^11.0 || ^12.0",
"illuminate/database": "^8.0 || ^9.0 || ^10.0 || ^11.0 || ^12.0"
},
"require-dev": {
"orchestra/testbench": "^6.0 || ^7.0 || ^8.0 || ^9.0 || ^10.0",
"phpunit/phpunit": "^9.6 || ^10.0 || ^11.0"
},
"autoload": {
"psr-4": {
"Ginkelsoft\\EncryptedSearch\\": "src/"
}
},
"autoload-dev": {
"psr-4": {
"Ginkelsoft\\EncryptedSearch\\Tests\\": "tests/",
"Tests\\": "tests/"
}
},
"extra": {
"laravel": {
"providers": [
"Ginkelsoft\\EncryptedSearch\\EncryptedSearchServiceProvider"
]
}
},
"require": {
"php": ">=8.2",
"illuminate/support": "^11.0|^12.0",
"illuminate/database": "^11.0|^12.0"
},
"require-dev": {
"orchestra/testbench": "^9.0",
"phpunit/phpunit": "^10.5|^11.0"
},
"autoload-dev": {
"psr-4": {
"Ginkelsoft\\EncryptedSearch\\Tests\\": "tests/",
"Tests\\": "tests/"
}
}
"minimum-stability": "stable",
"prefer-stable": true
}