Skip to content

Conversation

@xborder
Copy link
Contributor

@xborder xborder commented Dec 24, 2025

What's Changed

Flight JDBC driver does not support IPC compressed formats. This PR includes and shades arrow-compression when packing JDBC driver.
For zstd compression, it is excluding zstd-jni relocation since it needs JNI native methods to keep original location.

Closes #926.

@github-actions
Copy link

Thank you for opening a pull request!

Please label the PR with one or more of:

  • bug-fix
  • chore
  • dependencies
  • documentation
  • enhancement

Also, add the 'breaking-change' label if appropriate.

See CONTRIBUTING.md for details.

@xborder
Copy link
Contributor Author

xborder commented Dec 24, 2025

@aarshivv are you able to build the jdbc driver and test it against your server? I created a demo flight server compressing IPC stream and used DBeaver to load the driver but I'd like to validate this with someone else if possible

@jbonofre
Copy link
Member

I will do a quick test too.

@aarshivv
Copy link

@xborder , I tested it against my Rust-based Flight SQL server and it is working as expected for ZSTD compression.

However, I am getting the following error when the server uses LZ4_FRAME compression:

java.sql.SQLException: Error while executing SQL "select * from some_table": Arrow error: Invalid argument error: lz4 IPC compression requires the lz4 feature
        at org.apache.arrow.driver.jdbc.shaded.org.apache.calcite.avatica.Helper.createException(Helper.java:56)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.calcite.avatica.Helper.createException(Helper.java:41)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:228)
        at com.example.flightsql.ArrowFlightSqlClient.executeAndPrint(ArrowFlightSqlClient.java:108)
        at com.example.flightsql.ArrowFlightSqlClient.main(ArrowFlightSqlClient.java:186)
Caused by: org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.FlightRuntimeException: INTERNAL: Arrow error: Invalid argument error: lz4 IPC compression requires the lz4 feature

I confirmed that the shaded JAR includes the compression classes:
org/apache/arrow/driver/jdbc/shaded/org/apache/arrow/compression/Lz4CompressionCodec.class

@xborder
Copy link
Contributor Author

xborder commented Dec 24, 2025

@aarshivv could that be a server error? The error message seems to hint that

@aarshivv
Copy link

@xborder , I am sure that it is not a server error, I tried the same with other clients and it is working fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JDBC driver does not support compressed IPC streams (ZSTD, LZ4) [FlightSQL]

3 participants