-
-
Notifications
You must be signed in to change notification settings - Fork 6
Add Hive 4.2.0 and fix 4.0.1 OPA authorizer shaded jar #1356
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 19 commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
68f11c1
remive schematool fix
maltesander 078db15
cleanup and comments
maltesander a9ff8d8
remove 4.0.0 patches
maltesander f5530ee
wip - 4.0.1 working with jar rename
maltesander 40cc57a
fix 4.0.1 by relocating jline
maltesander 6afc47e
fixes & cleanup
maltesander 1ab6431
fix typo
maltesander b511e99
fix 3.1.3
maltesander d7d4c40
fix mvn mount cache
maltesander 7bc3a22
adapted changelog
maltesander eb1130f
Merge branch 'main' into fix-4.0.1-opa-authorizer
maltesander c8673ad
readd 4.0.0
maltesander 1d4af7d
readd schematool fix for 4.0.x
maltesander f2b1d10
adapt condition for 4.0.0
maltesander 93e8b50
add 4.2.0
maltesander cc38671
linter & changelog
maltesander 1e32fde
add set -e to hive build
maltesander 9f2ed0c
add http5 client to standalone-metastore for REST Catalog
maltesander 5166dda
readd schematool todo
maltesander 3f3222d
Merge remote-tracking branch 'origin/main' into fix-4.0.1-opa-authorizer
maltesander 086da0f
fix changelog
maltesander 82031ab
remove SHELL and set -e
maltesander 656081d
fix SC3014 and SC3015 reason
maltesander File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
37 changes: 37 additions & 0 deletions
37
...store-opa-authorizer/stackable/patches/v1.0.0/0001-Relocate-jline-in-shading-plugin.patch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| From e9b5503e9c952ab691952fb14724ac55a6890420 Mon Sep 17 00:00:00 2001 | ||
| From: Malte Sander <malte.sander.it@gmail.com> | ||
| Date: Mon, 24 Nov 2025 18:05:16 +0100 | ||
| Subject: Relocate jline in shading plugin | ||
|
|
||
| For Hive 4.0.1 the dependency loading is a mess. We explicitly shade jline to avoid a schematool dependency problem: | ||
|
|
||
| Exception in thread "main" java.lang.NoSuchMethodError: 'void org.jline.reader.impl.completer.StringsCompleter.<init>(org.jline.reader.Candidate[])' | ||
| at sqlline.SqlLineOpts.setOptionCompleters(SqlLineOpts.java:160) | ||
| at sqlline.Application.getCommandHandlers(Application.java:294) | ||
| at sqlline.SqlLine$Config.<init>(SqlLine.java:1946) | ||
| at sqlline.SqlLine.setAppConfig(SqlLine.java:1875) | ||
| at sqlline.SqlLine.<init>(SqlLine.java:229) | ||
| at org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.execSql(MetastoreSchemaTool.java:313) | ||
|
|
||
| This does not affect versions >= 4.1.x since the non shaded jar is used. | ||
| --- | ||
| hms-v4/pom.xml | 6 ++++++ | ||
| 1 file changed, 6 insertions(+) | ||
|
|
||
| diff --git a/hms-v4/pom.xml b/hms-v4/pom.xml | ||
| index 4ebeac4..b2940ab 100644 | ||
| --- a/hms-v4/pom.xml | ||
| +++ b/hms-v4/pom.xml | ||
| @@ -67,6 +67,12 @@ | ||
| </excludes> | ||
| </filter> | ||
| </filters> | ||
| + <relocations> | ||
| + <relocation> | ||
| + <pattern>org.jline</pattern> | ||
| + <shadedPattern>com.bosch.bdps.jline</shadedPattern> | ||
| + </relocation> | ||
| + </relocations> | ||
| </configuration> | ||
| </execution> | ||
| </executions> |
34 changes: 34 additions & 0 deletions
34
hive/stackable/patches/4.2.0/0001-Include-Postgres-driver.patch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| From f7913f5ee840dc277301fcb0f18e04be7849ab5c Mon Sep 17 00:00:00 2001 | ||
| From: Malte Sander <malte.sander.it@gmail.com> | ||
| Date: Tue, 25 Nov 2025 11:29:03 +0100 | ||
| Subject: Include Postgres driver | ||
|
|
||
| --- | ||
| standalone-metastore/metastore-server/pom.xml | 1 - | ||
| standalone-metastore/pom.xml | 1 - | ||
| 2 files changed, 2 deletions(-) | ||
|
|
||
| diff --git a/standalone-metastore/metastore-server/pom.xml b/standalone-metastore/metastore-server/pom.xml | ||
| index fa2418f205..0e24b3ebc9 100644 | ||
| --- a/standalone-metastore/metastore-server/pom.xml | ||
| +++ b/standalone-metastore/metastore-server/pom.xml | ||
| @@ -338,7 +338,6 @@ | ||
| <dependency> | ||
| <groupId>org.postgresql</groupId> | ||
| <artifactId>postgresql</artifactId> | ||
| - <optional>true</optional> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.eclipse.jetty</groupId> | ||
| diff --git a/standalone-metastore/pom.xml b/standalone-metastore/pom.xml | ||
| index bd6f2227b6..0bf9d09c0b 100644 | ||
| --- a/standalone-metastore/pom.xml | ||
| +++ b/standalone-metastore/pom.xml | ||
| @@ -435,7 +435,6 @@ | ||
| <groupId>org.postgresql</groupId> | ||
| <artifactId>postgresql</artifactId> | ||
| <version>${postgres.version}</version> | ||
| - <scope>runtime</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.httpcomponents</groupId> |
25 changes: 25 additions & 0 deletions
25
hive/stackable/patches/4.2.0/0002-Include-logging-dependencies.patch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| From 8a74bc78eb3ed664e698dd41be25107a495a018e Mon Sep 17 00:00:00 2001 | ||
| From: Malte Sander <malte.sander.it@gmail.com> | ||
| Date: Tue, 25 Nov 2025 11:30:24 +0100 | ||
| Subject: Include logging dependencies | ||
|
|
||
| --- | ||
| standalone-metastore/pom.xml | 5 +++++ | ||
| 1 file changed, 5 insertions(+) | ||
|
|
||
| diff --git a/standalone-metastore/pom.xml b/standalone-metastore/pom.xml | ||
| index 0bf9d09c0b..40bd4fcd83 100644 | ||
| --- a/standalone-metastore/pom.xml | ||
| +++ b/standalone-metastore/pom.xml | ||
| @@ -582,6 +582,11 @@ | ||
| <groupId>com.fasterxml.jackson.core</groupId> | ||
| <artifactId>jackson-databind</artifactId> | ||
| </dependency> | ||
| + <dependency> | ||
| + <!-- Optional log4j dependency to be able to use the XmlLayout --> | ||
| + <groupId>com.fasterxml.jackson.dataformat</groupId> | ||
| + <artifactId>jackson-dataformat-xml</artifactId> | ||
| + </dependency> | ||
| </dependencies> | ||
| <build> | ||
| <pluginManagement> |
26 changes: 26 additions & 0 deletions
26
hive/stackable/patches/4.2.0/0003-Fix-CVE-2024-36114.patch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| From 4095defdcf9e049dfa5a7c9b279760ea30615465 Mon Sep 17 00:00:00 2001 | ||
| From: Malte Sander <malte.sander.it@gmail.com> | ||
| Date: Tue, 25 Nov 2025 11:31:34 +0100 | ||
| Subject: Fix CVE-2024-36114 | ||
|
|
||
| --- | ||
| standalone-metastore/pom.xml | 6 ++++++ | ||
| 1 file changed, 6 insertions(+) | ||
|
|
||
| diff --git a/standalone-metastore/pom.xml b/standalone-metastore/pom.xml | ||
| index 40bd4fcd83..34eb45fb96 100644 | ||
| --- a/standalone-metastore/pom.xml | ||
| +++ b/standalone-metastore/pom.xml | ||
| @@ -157,6 +157,12 @@ | ||
| <version>${netty.version}</version> | ||
| <classifier>linux-x86_64</classifier> | ||
| </dependency> | ||
| + <!-- Mitigate CVE-2024-36114: See https://github.com/stackabletech/vulnerabilities/issues/834 --> | ||
| + <dependency> | ||
| + <groupId>io.airlift</groupId> | ||
| + <artifactId>aircompressor</artifactId> | ||
| + <version>0.27</version> | ||
| + </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.orc</groupId> | ||
| <artifactId>orc-core</artifactId> |
40 changes: 40 additions & 0 deletions
40
hive/stackable/patches/4.2.0/0004-Add-httpcomponents.core5-to-hive-standalone-metastor.patch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| From 897f67b3bd6f200affe80423a37c7de202f13782 Mon Sep 17 00:00:00 2001 | ||
| From: Malte Sander <malte.sander.it@gmail.com> | ||
| Date: Tue, 25 Nov 2025 14:44:35 +0100 | ||
| Subject: Add httpcomponents.core5 to hive-standalone-metastore | ||
|
|
||
| Required for the REST Catalog. | ||
| --- | ||
| standalone-metastore/metastore-rest-catalog/pom.xml | 5 +++++ | ||
| standalone-metastore/pom.xml | 2 ++ | ||
| 2 files changed, 7 insertions(+) | ||
|
|
||
| diff --git a/standalone-metastore/metastore-rest-catalog/pom.xml b/standalone-metastore/metastore-rest-catalog/pom.xml | ||
| index c1692e7ba2..ad6cecb535 100644 | ||
| --- a/standalone-metastore/metastore-rest-catalog/pom.xml | ||
| +++ b/standalone-metastore/metastore-rest-catalog/pom.xml | ||
| @@ -26,6 +26,11 @@ | ||
| <iceberg.version>1.9.1</iceberg.version> | ||
| </properties> | ||
| <dependencies> | ||
| + <dependency> | ||
| + <groupId>org.apache.httpcomponents.core5</groupId> | ||
| + <artifactId>httpcore5</artifactId> | ||
| + <version>${httpcomponents5.core.version}</version> | ||
| + </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.hive</groupId> | ||
| <artifactId>hive-standalone-metastore-server</artifactId> | ||
| diff --git a/standalone-metastore/pom.xml b/standalone-metastore/pom.xml | ||
| index 34eb45fb96..94b883b801 100644 | ||
| --- a/standalone-metastore/pom.xml | ||
| +++ b/standalone-metastore/pom.xml | ||
| @@ -117,6 +117,8 @@ | ||
| <slf4j.version>1.7.30</slf4j.version> | ||
| <httpcomponents.core.version>4.4.13</httpcomponents.core.version> | ||
| <httpcomponents.client.version>4.5.13</httpcomponents.client.version> | ||
| + <!-- Required for REST Catalog. Should be in sync with Iceberg. --> | ||
| + <httpcomponents5.core.version>5.3.1</httpcomponents5.core.version> | ||
| <pac4j-core.version>4.5.8</pac4j-core.version> | ||
| <nimbus-oauth.version>11.28</nimbus-oauth.version> | ||
| <jetty.version>9.4.57.v20241219</jetty.version> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| mirror = "https://github.com/stackabletech/hive.git" | ||
| base = "cb06ad72d609e51b6a3a38ccb120e34b4281067c" |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.