fix & impr (core + hql + cli): gateway permissions checking, cli rendering issues, default values for edges and vecs, custom return structures (#692)

## Description
<!-- Provide a brief description of the changes in this PR -->

## Related Issues
<!-- Link to any related issues using #issue_number -->

Closes #

## Checklist when merging to main
<!-- Mark items with "x" when completed -->

- [ ] No compiler warnings (if applicable)
- [ ] Code is formatted with `rustfmt`
- [ ] No useless or dead code (if applicable)
- [ ] Code is easy to understand
- [ ] Doc comments are used for all functions, enums, structs, and
fields (where appropriate)
- [ ] All tests pass
- [ ] Performance has not regressed (assuming change was not to fix a
bug)
- [ ] Version number has been updated in `helix-cli/Cargo.toml` and
`helixdb/Cargo.toml`

## Additional Notes
<!-- Add any additional information that would be helpful for reviewers
-->

<!-- greptile_comment -->

<h2>Greptile Overview</h2>

<h3>Greptile Summary</h3>


This PR merges the `dev` branch into `main`, introducing API key
verification, vector database enhancements, query generator
improvements, and various bug fixes.

## Major Changes
- **API Key Authentication**: Added optional API key verification using
SHA-256 hashing and constant-time comparison (feature-gated with
`api-key` flag)
- **Vector Core Enhancements**: Implemented `get_all_vectors` method
with level filtering and improved error messages
- **Query Generator**: Enhanced object literal construction, nested
struct handling, and closure variable resolution for complex query
patterns
- **Builtin Handlers**: Updated to use arena allocators for improved
memory management
- **Test Coverage**: Added comprehensive integration tests for API key
verification

## Issues Found
- **Critical**: `gateway.rs:178` uses `.unwrap()` on `api_key_hash`
which could panic if the field is unexpectedly None (though this should
be prevented by request validation when the feature is enabled)

<details><summary><h3>Important Files Changed</h3></summary>



File Analysis



| Filename | Score | Overview |
|----------|-------|----------|
| helix-db/src/helix_gateway/key_verification.rs | 4/5 | New API key
verification module with constant-time comparison using `subtle` crate,
comprehensive test coverage |
| helix-db/src/helix_gateway/gateway.rs | 3/5 | Added API key
verification to `post_handler`, uses `.unwrap()` which could panic,
follows OnceLock const block rule correctly |
| helix-db/src/protocol/request.rs | 5/5 | Added `api_key_hash` field to
Request struct, hashes incoming x-api-key header with SHA-256,
comprehensive tests added |
| helix-db/src/protocol/error.rs | 5/5 | Added `InvalidApiKey` error
variant with 403 HTTP status code, proper error handling and tests |
| helix-db/src/helix_engine/vector_core/vector_core.rs | 5/5 | Added
`get_all_vectors` method with optional level filtering, improved error
messages with uuid_str utility |
| helix-db/src/helixc/generator/queries.rs | 4/5 | Enhanced query
generator with object literal construction, nested struct handling, and
improved closure variable resolution |

</details>


</details>


<details><summary><h3>Sequence Diagram</h3></summary>

```mermaid
sequenceDiagram
    participant Client
    participant Gateway as Helix Gateway<br/>(post_handler)
    participant Request as Request Extractor<br/>(from_request)
    participant KeyVerify as key_verification<br/>(verify_key)
    participant WorkerPool as Worker Pool
    participant Response as HTTP Response

    Client->>Gateway: POST /{query_name}<br/>Header: x-api-key
    Gateway->>Request: Extract request
    
    alt api-key feature enabled
        Request->>Request: SHA-256 hash x-api-key header
        Request->>Request: Store hash in api_key_hash field
    else api-key feature disabled
        Request->>Request: Set api_key_hash = None
    end
    
    Request->>Gateway: Return Request object
    
    alt api-key feature enabled
        Gateway->>Gateway: unwrap() api_key_hash
        Gateway->>KeyVerify: verify_key(hash)
        KeyVerify->>KeyVerify: Constant-time comparison<br/>with stored hash
        
        alt Invalid key
            KeyVerify->>Gateway: Err(InvalidApiKey)
            Gateway->>Gateway: Log metrics event
            Gateway->>Response: 403 Forbidden
            Response->>Client: Error response
        else Valid key
            KeyVerify->>Gateway: Ok()
        end
    end
    
    Gateway->>WorkerPool: process(request)
    WorkerPool->>WorkerPool: Execute query
    WorkerPool->>Gateway: Result
    
    alt Success
        Gateway->>Gateway: Log success metrics
        Gateway->>Response: 200 OK with data
    else Error
        Gateway->>Gateway: Log error metrics
        Gateway->>Response: Error status code
    end
    
    Response->>Client: HTTP Response
```
</details>


<!-- greptile_other_comments_section -->

**Context used:**

- Rule from `dashboard` - When using OnceLock::new() in Rust code, wrap
it in a `const { ... }` block to satisfy clippy lints....
([source](https://app.greptile.com/review/custom-context?memory=7f3c5d9b-5a5e-41cd-9e7e-1992245f637c))

<!-- /greptile_comment -->

Author: xav-db

.github/workflows/db_tests.yml

@@ -11,7 +11,9 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest, windows-latest, macos-latest]
-        
+    env:
+      HELIX_API_KEY: "12345678901234567890123456789012"
+
     steps:
     - uses: actions/checkout@v4
 
@@ -33,3 +35,13 @@ jobs:
       run: | 
         cd helix-db
         cargo test --release --lib -- --skip concurrency_tests
+
+    - name: Run dev instance tests
+      run: | 
+        cd helix-db
+        cargo test --release --lib --features dev-instance -- --skip concurrency_tests
+
+    - name: Run production tests
+      run: |
+        cd helix-db
+        cargo test --release --lib --features production -- --skip concurrency_tests    

.gitignore

@@ -8,4 +8,5 @@ hack/
 .DS_Store
 /tmp
 /data
-claude.md
+claude.md
+.cargo/

Cargo.lock

@@ -1,6 +1,6 @@
 [package]
 name = "helix-cli"
-version = "2.1.1"
+version = "2.1.2"
 edition = "2024"
 
 [dependencies]

helix-cli/Cargo.toml

@@ -3,7 +3,8 @@ use crate::docker::DockerManager;
 use crate::metrics_sender::MetricsSender;
 use crate::project::{ProjectContext, get_helix_repo_cache};
 use crate::utils::{
-    copy_dir_recursive_excluding, helixc_utils::collect_hx_files, print_status, print_success,
+    copy_dir_recursive_excluding, diagnostic_source, helixc_utils::collect_hx_files, print_status,
+    print_success,
 };
 use eyre::Result;
 use std::time::Instant;
@@ -24,8 +25,7 @@ use helix_db::{
         },
     },
 };
-use std::fmt::Write;
-use std::fs;
+use std::{fmt::Write, fs};
 
 // Development flag - set to true when working on V2 locally
 const DEV_MODE: bool = cfg!(debug_assertions);
@@ -329,7 +329,7 @@ fn compile_helix_files(
     };
 
     // Run static analysis
-    let mut analyzed_source = analyze_source(source)?;
+    let mut analyzed_source = analyze_source(source, &content.files)?;
 
     // Read and set the config from the instance workspace
     analyzed_source.config = read_config(instance_src_dir)?;
@@ -371,7 +371,7 @@ fn parse_content(content: &Content) -> Result<Source> {
 
 /// Runs the static analyzer on the parsed source to catch errors and generate diagnostics if any.
 /// Otherwise returns the generated source object which is an IR used to transpile the queries to rust.
-fn analyze_source(source: Source) -> Result<GeneratedSource> {
+fn analyze_source(source: Source, files: &[HxFile]) -> Result<GeneratedSource> {
     if source.schema.is_empty() {
         let error = crate::errors::CliError::new("no schema definitions found in project")
             .with_hint("add at least one schema definition like 'N::User { name: String }' to your .hx files");
@@ -384,7 +384,8 @@ fn analyze_source(source: Source) -> Result<GeneratedSource> {
         let mut error_msg = String::new();
         for diag in diagnostics {
             let filepath = diag.filepath.clone().unwrap_or("queries.hx".to_string());
-            error_msg.push_str(&diag.render(&source.source, &filepath));
+            let snippet_src = diagnostic_source(&filepath, files, &source.source);
+            error_msg.push_str(&diag.render(snippet_src.as_ref(), &filepath));
             error_msg.push('\n');
         }
         return Err(eyre::eyre!("Compilation failed:\n{error_msg}"));

helix-cli/src/commands/build.rs

@@ -68,7 +68,7 @@ fn validate_project_syntax(project: &ProjectContext) -> Result<()> {
     }
 
     // Run static analysis to catch validation errors
-    analyze_source(source)?;
+    analyze_source(source, &content.files)?;
 
     print_success("All queries and schema are valid");
     Ok(())

helix-cli/src/commands/check.rs

@@ -41,7 +41,7 @@ pub async fn run(output_dir: Option<String>, path: Option<String>) -> Result<()>
     }
 
     // Run static analysis to catch validation errors
-    let generated_source = analyze_source(source)?;
+    let generated_source = analyze_source(source, &content.files)?;
 
     // Generate Rust code
     let output_dir = output_dir

helix-cli/src/commands/compile.rs

@@ -1,11 +1,26 @@
 use crate::errors::CliError;
 use color_eyre::owo_colors::OwoColorize;
 use eyre::{Result, eyre};
-use std::fs;
-use std::path::Path;
+use helix_db::helixc::parser::types::HxFile;
+use std::{borrow::Cow, fs, path::Path};
 
 const IGNORES: [&str; 3] = ["target", ".git", ".helix"];
 
+/// Resolve the source text to use when rendering diagnostics.
+pub fn diagnostic_source<'a>(
+    filepath: &str,
+    files: &'a [HxFile],
+    fallback: &'a str,
+) -> Cow<'a, str> {
+    if let Some(src) = files.iter().find(|file| file.name == filepath) {
+        Cow::Borrowed(src.content.as_str())
+    } else if let Ok(contents) = fs::read_to_string(filepath) {
+        Cow::Owned(contents)
+    } else {
+        Cow::Borrowed(fallback)
+    }
+}
+
 /// Copy a directory recursively without any exclusions
 pub fn copy_dir_recursively(src: &Path, dst: &Path) -> Result<()> {
     if !src.is_dir() {
@@ -233,8 +248,7 @@ pub mod helixc_utils {
             types::{Content, HxFile, Source},
         },
     };
-    use std::fs;
-    use std::path::Path;
+    use std::{fs, path::Path};
 
     /// Collect all .hx files from queries directory and subdirectories
     pub fn collect_hx_files(root: &Path, queries_dir: &Path) -> Result<Vec<std::fs::DirEntry>> {
@@ -302,13 +316,14 @@ pub mod helixc_utils {
     }
 
     /// Analyze source for validation (similar to build.rs)
-    pub fn analyze_source(source: Source) -> Result<GeneratedSource> {
+    pub fn analyze_source(source: Source, files: &[HxFile]) -> Result<GeneratedSource> {
         let (diagnostics, generated_source) =
             analyze(&source).map_err(|e| eyre::eyre!("Analysis error: {}", e))?;
 
         if !diagnostics.is_empty() {
             // Format diagnostics properly using the helix-db pretty printer
-            let formatted_diagnostics = format_diagnostics(&diagnostics, &generated_source.src);
+            let formatted_diagnostics =
+                format_diagnostics(&diagnostics, &generated_source.src, files);
             return Err(eyre::eyre!(
                 "Compilation failed with {} error(s):\n\n{}",
                 diagnostics.len(),
@@ -323,6 +338,7 @@ pub mod helixc_utils {
     fn format_diagnostics(
         diagnostics: &[helix_db::helixc::analyzer::diagnostic::Diagnostic],
         src: &str,
+        files: &[HxFile],
     ) -> String {
         let mut output = String::new();
         for diagnostic in diagnostics {
@@ -332,7 +348,8 @@ pub mod helixc_utils {
                 .clone()
                 .unwrap_or("queries.hx".to_string());
 
-            output.push_str(&diagnostic.render(src, &filepath));
+            let snippet_src = super::diagnostic_source(&filepath, files, src);
+            output.push_str(&diagnostic.render(snippet_src.as_ref(), &filepath));
             output.push('\n');
         }
         output

helix-cli/src/utils.rs

@@ -1,7 +1,6 @@
-
 // DEFAULT CODE
 use helix_db::helix_engine::traversal_core::config::Config;
 
 pub fn config() -> Option<Config> {
     None
-}
+}

helix-container/src/queries.rs

@@ -1,6 +1,6 @@
 [package]
 name = "helix-db"
-version = "1.1.1"
+version = "1.1.2"
 edition = "2024"
 description = "HelixDB is a powerful, open-source, graph-vector database built in Rust for intelligent data storage for RAG and AI."
 license = "AGPL-3.0"
@@ -57,6 +57,8 @@ polars = { version = "0.46.0", features = [
     "lazy",
     "json",
 ], optional = true }
+subtle = "2.6.1"
+sha_256 = "=0.1.1"
 
 [dev-dependencies]
 rand = "0.9.0"
@@ -73,7 +75,7 @@ compiler = ["pest", "pest_derive"]
 
 # vector features
 cosine = []
-
+api-key = []
 build = ["compiler"]
 vectors = ["cosine", "url"]
 server = ["build", "compiler", "vectors", "reqwest"]
@@ -82,13 +84,4 @@ bench = ["polars"]
 dev = ["debug-output", "server", "bench"]
 dev-instance = []
 default = ["server"]
-
-# benches/tests
-[[test]]
-name = "bm25_benches"
-path = "benches/bm25_benches.rs"
-
-[[test]]
-name = "hnsw_benches"
-path = "benches/hnsw_benches.rs"
-
+production = ["api-key","server"]