Skip to content

Update dependency org.questdb:questdb to v9.2.2 #319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update dependency org.questdb:questdb to v9.2.2 #319

wants to merge 1 commit into from

Conversation

@renovate
Copy link

@renovate renovate bot commented Oct 3, 2025 •
edited
Loading

This PR contains the following updates:

Package Change Age Confidence
org.questdb:questdb (source) 9.0.3 -> 9.2.2 age confidence

Release Notes

questdb/questdb (org.questdb:questdb)

v9.2.2

Compare Source

QuestDB 9.2.2 is another stability release, primarily fixing some query snags. With this, we've packaged a performance improvement for vertically-scaled machines, and some new SQL functions.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Breaking changes 💥

  • We've removed the legacy txn scoreboard (v1).
    • Using the new scoreboard (v2) has been the default for some time now.
    • If you have been manually overriding this config option (cairo.txn.scoreboard.format=1), it will no longer take effect, but the instance will boot and function as normal.
  • MANUAL PERIOD views will now only be refreshed by manual trigger, and not also automatically when the time period ends.
    • This was always the intended behaviour; it is unexpected that a MANUAL view should refresh automatically.

SQL

New weighted statistics functions

  • weighted_avg(DD) - computes the weighted average based on a 'value' column and a 'weights' column.
    • A straightforward weighted average calculation.
  • weighted_stddev_rel(DD) - computes the weighted standard deviation based a 'value' column and a 'reliability weights' column.
    • Reliability weights are based on how trusted or reliable the particular value should be treated.
  • weighted_stddev_freq(DD) - computes the weighted standard deviation based on a 'value' column and a 'frequency weights' column.
    • Frequency weights are based on the frequency of a particular sample occurring in the dataset.

Changelist

Full Changelog: questdb/questdb@9.2.1...9.2.2

v9.2.1

Compare Source

QuestDB 9.2.1 is a stability release, bringing a number of fixes, and some key performance enhancements, including markout horizon CROSS JOIN upgrades, and improvements to SQL query latencies across the board.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Highlights

Materialized Views

Materialized Views now have simplified syntax for building views with only-complete buckets, by specifying that the refresh period should match the sample by interval.

CREATE MATERIALIZED VIEW trades_daily_prices
REFRESH PERIOD (SAMPLE BY INTERVAL) AS
SELECT
  timestamp,
  symbol,
  avg(price) AS avg_price
FROM trades
SAMPLE BY 1d ALIGN TO CALENDAR TIME ZONE 'Europe/London';

In addition, PERIOD views now supported intervals in seconds.

CROSS JOIN

We've released a new optimisation for certain CROSS JOIN queries, specifically for markout analysis. This is a new query hint, which drastically speeds up building a market horizon table. Here is an example:

CREATE TABLE orders (
      ts TIMESTAMP,
      price DOUBLE
  ) timestamp(ts) PARTITION BY DAY;

INSERT INTO orders
  SELECT
      generate_series as timestamp,
      rnd_double() * 10.0 + 5.0
      FROM generate_series('2025-01-02', '2025-01-02T00:10', '200u');

WITH
  offsets AS (
    SELECT sec_offs, 10_000_000 * sec_offs usec_offs 
    FROM (SELECT x-61 AS sec_offs FROM long_sequence(121))
  )
  SELECT /*+ markout_horizon(orders offsets) */ sum(price) FROM (SELECT * FROM (
    SELECT price, ts + usec_offs AS timestamp
    FROM orders CROSS JOIN offsets
    ORDER BY ts + usec_offs
  ) TIMESTAMP(timestamp));

On an r7a.4xlarge instance:

Metric Without Hint With Hint
Query time 135s 17s
Additional memory usage 8.4 GB 0.1 GB

This is part of a series of features focused around optimising post-trade analysis, and driven directly by user feedback, so keep it coming!

Performance

  • Reduction in GC overhead for executing high-cardinality GROUP BY/SAMPLE BY queries.
  • count_distinct() has been sped up, up to 2x based on standard benchmarks.
  • Reduced memory usage for reads and writes over HTTP.
  • General SQL query latency reduction by optimising munmap usage:
    • This is opt-in, and can be enabled by setting cairo.file.async.unmap.enabled=true

SQL

  • New files(s) and glob(s) for scanning and filtering the server's filesystem.
  • New glob(Ss) function, an alternative to LIKE and ~ that uses glob-style syntax.
  • New first_not_null(array) and last_not_null(array) group by functions.
  • New API endpoint for validating SQL queries: /api/v1/sql/validate.

Changelist

New Contributors

Full Changelog: questdb/questdb@9.2.0...9.2.1

v9.2.0

Compare Source

✨ QuestDB 9.2: Native DECIMAL Type

QuestDB 9.2.0 brings the long-awaited DECIMAL type, a bespoke decimal type, declared using DECIMAL(precision, scale). Under the hood, the database automatically selects an optimal storage size, from DECIMAL8 (1 byte) up to DECIMAL256 (32 bytes). All of our usual arithmetical operations are supported out-of-the-box, making it easy to adopt and avoid precision loss.

With this new DECIMAL type, and our recently released support for nanosecond timestamps (TIMESTAMP_NS), QuestDB has never been a better fit for capital markets!

This release also brings upgrades to our already-fast ASOF JOIN queries, with a new Dense algorithm optimised for distant row matching. This marks our fourth ASOF JOIN algorithm, helping you to ensure your temporal join queries stay performant, regardless of your underlying data distribution.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Highlights

DECIMAL(precision, scale)

High-precision arithmetic

DECIMAL allows for arithmetic without precision loss, and only a slightly performance cost (2x for DECIMAL64 vs DOUBLE); aoperations which would silently lose precision with DOUBLE will instead warn the user, preserving data integrity.

-- With DOUBLE:
SELECT 0.1 + 0.2; -- returns: 0.30000000000000004 

-- With DECIMAL:
SELECT 0.1m + 0.2m;  -- returns: 0.3

Configurable scale

Configure precision and scale directly, without worrying about the underlying data type, with support for up to 76 digits!

CREATE TABLE transactions (
    id LONG,
    amount DECIMAL(14, 2),      -- Up to 999,999,999,999.99
    tax_rate DECIMAL(5, 4),      -- Up to 9.9999 (e.g., 0.0875 for 8.75%)
    quantity DECIMAL(10, 3),     -- Up to 9,999,999.999
    timestamp TIMESTAMP
) timestamp(timestamp);

API compatibility

DECIMAL is supported over PG Wire, ILP and HTTP APIs, with parquet support coming very soon.

Use cases

  • Preserving integrity of data in ledgers, simplifying regulatory compliance
  • Easy storage for cryptocurrency data, ordinarily requiring at least 18 digits of precision.
  • No need to work around integer-scaling or integer signedness issues.

See the decimal concepts documentation for more information.

Dense ASOF JOIN

QuestDB 9.2.0 introduces a new 'dense' ASOF JOIN algorithm, optimised for datasets where the matching rows are likely to be far apart. This complements the existing ASOF JOIN algorithms to ensure that your queries can achieve the best performance regardless of your data distribution.

The choice of which algorithm to use is important, and explained in detail in the optimizer hints documentation.

Here is a summary of the algorithms:

  • Fast; optimised for closely matching rows, using a binary-search and backwards scans.
  • Memoized; the same as Fast, but more tolerant of occassional distant matches.
  • Light; optimised when the RHS table has a highly selective filter, using forward-only scan on RHS table.
  • Dense; optimised for distant matches, using a binary-search, bidirectional scans and memoized results.

We will continue to iterate on this feature in conjunction with the upcoming performance optimisations for markout analysis.

Changelist

Full Changelog: questdb/questdb@9.1.1...9.2.0

v9.1.1

Compare Source

QuestDB 9.1.1

QuestDB 9.1.1 focuses on data interoperability, nanosecond precision, and analytical performance.
This release introduces Parquet export, a faster ASOF JOIN for sparse symbol data, and several engine optimizations improving both query throughput and stability.

Highlights

Data Interoperability
  • Parquet export: COPY TO now supports exporting query results directly to Parquet format for seamless integration with data-lake and analytics tools such as Spark, Snowflake, and Athena (#​6008).
  • Optimized Parquet import: read_parquet() recognizes timestamp-sorted data, significantly accelerating ingest for large, time-ordered datasets (#​6079).
Precision
  • Nanosecond precision ingestion: The Java ILP client now supports nanosecond timestamps, aligning with QuestDB’s highest-precision timekeeping (#​6220).
  • Synthetic data generation: New rnd_symbol_zipfn() function produces high-cardinality symbol distributions following a Zipf / Power Law — ideal for testing realistic workloads (#​6227).
Performance
  • ASOF JOIN optimization: Major speedup for joins on low-frequency symbols via a memory-efficient lookup strategy that reuses symbol search results (#​6208, #​6333).
  • Query engine improvements:
    • Parallel count_distinct() now uses a lazy merge strategy, reducing memory usage (#​6268).
    • Expanded fast path optimization for ordered and limited single-column queries (#​6272).
    • Parallel SAMPLE BY execution for queries combining timestamps and aggregates (#​6275).
    • Disabled column pre-touch in parallel filters, improving cache efficiency (#​6280).

🧱 Parquet Export

QuestDB now supports exporting tables or queries to Parquet format using the COPY command — enabling direct data exchange with data lakes and analytics pipelines.

-- export an entire table to parquet, with same partitioning as original
COPY 'table_name' TO 'folder_name' WITH FORMAT PARQUET;

-- export a subquery to parquet with new partitioning
COPY (SELECT * FROM trades WHERE timestamp IN today()) TO 'folder_name' WITH FORMAT PARQUET PARTITION_BY MONTH;

Users can also parquet data ad-hoc using /exp?fmt=parquet (and omitting COPY).

Benefits

  • Efficient columnar compression and encoding
  • Open, widely supported format
  • Perfect for interoperability with modern data ecosystems

🧮 New SQL Functions

  • rnd_symbol_zipfn() – generates high-cardinality symbols following a Power Law distribution (#​6227).
  • last(array) – returns the last element from an array expression (#​6291).
  • Nanosecond support – added new nanos functions and improved CAST behavior for date types (#​6247).

⚙️ SQL & Query Engine

  • Recognize timestamp-sorted data in read_parquet() for faster imports (#​6079).
  • Fixed invalid column errors in GROUP BY on joined tables (#​6275, #​6332).
  • Improved GROUP BY and SAMPLE BY to avoid duplicate group keys (#​6275).
  • Fixed incorrect results from SAMPLE BY with mixed timestamp / aggregate expressions (#​6254).
  • Corrected handling of negative offsets in SAMPLE BY (#​6306).
  • Fixed JIT compilation for WHERE sym1 = sym2 filters (#​6321).
  • Fixed overflow in materialized view refresh interval and step calculations (#​6233, #​6258).
  • Interval constants can now be applied as time intrinsics (#​6263).
  • Prevented critical-level logs on group-by query cancellation (#​6235).
  • Fixed NullPointerException issues in window and optimizer code paths (#​6264, #​6265).

📥 Ingestion (ILP)

  • Java ILP client now supports nanosecond precision timestamps (#​6220).
  • Fixed flaky LineHttpSenderLoggingTest stability issue (#​6315).

🧩 Core & Operations

Resource & Reliability
  • Added resource pool tracing to help detect memory leaks during development and testing (#​6234).
  • Fixed potential connection leaks on Windows (#​6243).
  • Fixed index error after column addition and truncation of non-partitioned tables (#​6319).
  • Fixed NullPointerException in ColumnPurgeJob (#​6274).
  • Improved reliability under high connection load (#​6334).
  • Added debug logging to WAL purge operations (#​6336).
HTTP Server
  • Relaxed cookie parser for legacy HTTP 1.0 and ANSI C timestamp formats while maintaining RFC 2616 compliance (#​6290).
Web Console
  • Updated to Web Console v1.1.4 (#​6309).
  • Fixed issue preventing configuration override on upgrade (#​6240).

⚡ Performance & Stability

  • Disabled column pre-touch in parallel filters for better cache utilization (#​6280).
  • Added new GroupByArraySink for optimized aggregation (#​6237).
  • Improved parallel group-by performance and stability (#​6303).
  • Optimized count_distinct() memory usage via lazy merge (#​6268).
  • Faster single-column ordered queries with expanded fast-path coverage (#​6272).

🛠 Build & Infrastructure

  • Bumped gosu in Dockerfile to address CVE-2023-28642 (#​6252).
  • Added license field to Rust crates (#​6284).
  • CI pinned to macOS 14 for consistent builds (#​6314).
  • Extract site config files even if site extraction is disabled (#​6281).
  • Moved CI coverage jobs to Hetzner (#​6308).
  • Improved Windows service wrapper (#​6229).

👩‍💻 Developer Experience

  • Added infrastructure for PostgreSQL wire protocol Golang tests (#​6298).
  • Enhanced test robustness for nd-array memory leak detection (#​6297).
  • Prevented authorization errors immediately after table creation (#​6337).

🌟 New Contributors

🔗 Full Changelog

Compare 9.1.0 → 9.1.1 on GitHub

✅ Summary

QuestDB 9.1.1 extends the database’s analytical performance and interoperability, introducing Parquet export, nanosecond ingestion, and numerous query engine optimizations.
This release strengthens QuestDB’s position as a high-performance, open, and developer-friendly time-series database for real-world workloads.

What's Changed

v9.1.0

Compare Source

QuestDB 9.1.0 is now available. It is a feature-rich release bringing nanosecond timestamp data type, performance and stability improvements.

Highlights

  • Nanosecond timestamps: New TIMESTAMP_NS type adds nanosecond precision to time-series data (#​5685).
  • Continuous profiling: Integrated async-profiler lets you capture CPU and memory flame graphs continuously or on demand (#​6150).
  • JOIN improvements: Support for all JOIN types including RIGHT OUTER and FULL OUTER, plus dramatically faster ASOF JOINs on indexed columns (#​6158, #​6076).
  • New SQL functions: Added timestamp and long window functions, plus visibility into symbol table size via SHOW COLUMNS (#​6130, #​6128).
  • Performance wins:
    • Symbol map auto-scaling: Eliminates ingestion bottlenecks from fixed symbol capacity. Now millions of distinct values ingest in seconds instead of hours.
    • Faster IN (...) queries on large lists (breaking change 💥).

Nanosecond timestamp support 🛡️

QuestDB now supports nanosecond precision with a new data type: TIMESTAMP_NS.

  • The existing TIMESTAMP type remains microsecond precision and is unchanged.
  • TIMESTAMP_NS can be used in all contexts where TIMESTAMP is valid (columns, functions, joins, ordering, etc.).
  • Migration: No changes are required for existing tables. They will continue to use TIMESTAMP (microsecond).
  • Rollback compatibility:
    • If you do not create tables with TIMESTAMP_NS, you can safely roll back to QuestDB 9.0.3 or earlier.
    • If you do create tables with TIMESTAMP_NS, older versions will not recognize this type. Such tables must be dropped before rolling back.
  • Drivers and client libraries: TIMESTAMP_NS is supported by existing QuestDB drivers (JDBC, Go, Python, etc.), provided you are on a recent release of the driver. No protocol upgrade or client code changes are required.

Symbol map auto-scaling (opt-in)

Symbol column map capacity now scales automatically with the number of distinct values, removing one of the biggest friction points in high-ingestion workloads.

  • Before: symbol map capacity was fixed at table creation.

    • If undersized, ingestion performance could degrade by up to 1000×, often making ingestion practically impossible.
    • For example, on default settings the system would bog down at ~1,000,000 unique symbols, taking several hours to ingest that data (unless the user had pre-sized the symbol map correctly).

  • Now: with auto-scaling enabled, symbol capacity grows dynamically:

    • ~2,000,000 distinct values ingested in 2.5 seconds (vs hours).
    • ~40,000,000 distinct values ingested in 2.5 minutes (vs never finishing before).

  • Backward compatibility:

    • Auto-scaling is opt-in to preserve compatibility.
    • Enabling it stores metadata in a new structure that older versions cannot read.
    • If rollback is required, data does not need to be dropped, but some manual file renaming steps are necessary. Users can contact QuestDB support for guidance.

  • Opt-in instructions:

Enable via configuration property, system's restart is not required:

cairo.auto.scale.symbol.capacity=true

Continuous profiling support (opt-in)

QuestDB now ships with the Async Profiler integrated out-of-the-box. This dramatically reduces friction when diagnosing performance issues such as high CPU usage or queries that appear slow.

  • Ease of use: the profiler is bundled, no manual installation required.
  • Opt-in: profiling has no impact on performance unless enabled.
  • Performance impact:
    • 0% overhead when not in use.
    • ~10% overhead when the profiler is actively connected.
  • Modes of use:
    1. Start with profiling enabled – less common, but can help during development or long-running load tests.
    2. Attach to a running instance – the most useful mode, allows “catching the system in the act” when issues are intermittent or difficult to reproduce.
Examples

Start a 30s CPU profiling session and generate an interactive HTML flamegraph:

questdb.sh profile -- -e cpu -d 30 -f /tmp/cpu.html
image

SQL & Query Engine

  • Nanosecond timestamp support for ultra-high resolution time-series workloads (#​5685).
  • New window functions for timestamp and long data types (#​6130).
  • Symbol table size visibility added to SHOW COLUMNS (#​6128).
  • Optimized IN (...) comparisons on long lists — breaking change 💥 (#​6109).
  • Enhanced JOIN execution with multiple improvements (#​6076).
  • Faster ASOF JOIN on indexed symbol columns (#​6158).
  • Vectorized avg(short) and improved sum() functions memory usage (#​6166).
  • GROUP BY queries now rewrite trivial expressions over the same column for better performance (#​6043).
  • Improved null-handling in INSERT INTO SELECT queries with implicit casting (#​5586).
  • Consistent behavior of EXPLAIN command (#​6200).
  • Fixed NPE in DISTINCT queries with dependent columns (#​6080).
  • Fixed compilation error for symbol column to symbol cast in GROUP BY queries (#​6072).
  • Fixed nanosecond precision issues in ASOF JOIN TOLERANCE and CSV imports (#​6183).
  • Prevented multiple DECLARE statements in single query block (#​6161).
  • Fixed rare assertion error after metadata changes (#​6195).

Ingestion (ILP)

  • Improved handling of symbol ingestion with double column conversion (#​6143).
  • Retry logic fixed when fetching server protocol version (#​6099).
  • Support for configuring multiple possible database endpoints (#​6074).
  • Improved connection handling — now avoids closing connections on protocol errors (#​6190).

Core & Operations

Continuous profiling support

QuestDB now integrates the Async Profiler for low-overhead performance diagnostics.

  • Modes:
    • Continuous profiling at startup (questdb.sh start -p)
    • On-demand attach to running instances for CPU or allocation traces
  • Outputs:
    • Interactive HTML flame graphs
    • JFR (Java Flight Recorder) output (#​6150)
Threading and resource usage
  • Materialized view refreshes now run on a dedicated thread pool (#​6111)
Reliability fixes
  • Fixed rare index reader memory error on partition drop (#​6087)
  • Fixed suspended table showing wrong writerTxn (#​6090)
  • Fixed table read timeout after non-WAL drop/re-create (#​6095)
  • Fixed rare suspension on index drop (#​6147)
  • Fixed misleading “table locked” error appearing before write failures (#​6152)
  • Fixed failure handling on column renames (#​5752)
  • Fixed spurious “cannot mmap” errors (#​6104)
  • Fixed incorrect error handling that could leave table reader in an inconsistent state (#​6108)
  • Fixed memory tags used in Path (#​6124)
  • Fixed assertion exception in writer triggered by OS errors (#​6204)

Breaking change 💥

IN list queries now use hash-based lookups instead of binary search, delivering significant performance gains for large lists.

  • Performance: Faster execution for queries with thousands of IN values.
  • Configuration: JIT compilation is skipped for lists larger than cairo.sql.jit.max.in.list.size.threshold (default: 10,000).
  • Behaviour: Results remain equivalent, but execution order and memory usage may differ in edge cases.

Migration guidance:

  • No SQL syntax changes are required
  • If you notice regressions for small lists or need JIT for large lists, adjust cairo.sql.jit.max.in.list.size.threshold.
  • Benchmark your workloads to tune the threshold for optimal results.
  • Expect faster performance for large IN lists out of the box.

This change matters most for queries with very large IN lists (thousands of values), often seen in data filtering, batch imports, or security/ACL checks. These workloads will now execute much faster by default.

What's Changed

New Contributors

Full Changelog: questdb/questdb@9.0.3...9.1

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@renovate renovate bot force-pushed the renovate/org.questdb-questdb-9.x branch from f04eafe to b78c53f Compare October 10, 2025 21:10
@renovate renovate bot force-pushed the renovate/org.questdb-questdb-9.x branch from b78c53f to a243acb Compare October 21, 2025 14:08
@renovate renovate bot force-pushed the renovate/org.questdb-questdb-9.x branch from a243acb to 3578c1c Compare November 3, 2025 19:40
@renovate renovate bot changed the title Update dependency org.questdb:questdb to v9.1.0 Update dependency org.questdb:questdb to v9.1.1 Nov 3, 2025
@renovate renovate bot force-pushed the renovate/org.questdb-questdb-9.x branch from 3578c1c to cc81fda Compare November 14, 2025 00:03
@renovate renovate bot changed the title Update dependency org.questdb:questdb to v9.1.1 Update dependency org.questdb:questdb to v9.2.0 Nov 14, 2025
@renovate renovate bot force-pushed the renovate/org.questdb-questdb-9.x branch from cc81fda to 34cc8bd Compare November 25, 2025 17:32
@renovate renovate bot changed the title Update dependency org.questdb:questdb to v9.2.0 Update dependency org.questdb:questdb to v9.2.1 Nov 25, 2025
@renovate renovate bot force-pushed the renovate/org.questdb-questdb-9.x branch from 34cc8bd to f8e98c5 Compare December 1, 2025 16:14
@renovate renovate bot changed the title Update dependency org.questdb:questdb to v9.2.1 Update dependency org.questdb:questdb to v9.2.2 Dec 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant