Skip to content

Commit ad2bbbf

Browse files
dervoetidervoeti
authored
refactor(testing-tools): split into multiple images, switch to UBI, remove unnecessary components (#1354)
* feat: split testing tools into multiple images, switch to UBI as base, remove unnecessary components * chore: changelog * chore: pre-commit fix
1 parent fbb084f commit ad2bbbf

File tree

15 files changed

+184
-266
lines changed

15 files changed

+184
-266
lines changed

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ All notable changes to this project will be documented in this file.
1313

1414
- airflow: Extend list of providers for 3.0.6 ([#1336])
1515
- airflow: Bump celery version to 5.5.3 for Airflow 3.x ([#1343]).
16+
- testing-tools: refactoring: Split image into multiple images, remove unnecessary components and switch to UBI as base image ([#1354]).
1617

1718
### Removed
1819

@@ -22,6 +23,7 @@ All notable changes to this project will be documented in this file.
2223
[#1337]: https://github.com/stackabletech/docker-images/pull/1337
2324
[#1343]: https://github.com/stackabletech/docker-images/pull/1343
2425
[#1340]: https://github.com/stackabletech/docker-images/pull/1340
26+
[#1354]: https://github.com/stackabletech/docker-images/pull/1354
2527

2628
## [25.11.0] - 2025-11-07
2729

‎testing-tools/Dockerfile‎

Lines changed: 45 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,14 @@
11
# syntax=docker/dockerfile:1.16.0@sha256:e2dd261f92e4b763d789984f6eab84be66ab4f5f08052316d8eb8f173593acf7
22
# check=error=true
33

4-
# Manifest list digest because of multi architecture builds ( https://www.redhat.com/architect/pull-container-image#:~:text=A%20manifest%20list%20exists%20to,system%20on%20a%20specific%20architecture )
5-
# https://hub.docker.com/_/python/tags
6-
# In Docker Hub, open up the tag and look for Index Digest. Otherwise do:
7-
# docker pull python:3.12-slim-bullseye and see the digest that appears in the output.
8-
FROM python:3.12-slim-bullseye@sha256:411fa4dcfdce7e7a3057c45662beba9dcd4fa36b2e50a2bfcd6c9333e59bf0db
4+
# Find the latest version at https://catalog.redhat.com/en/software/containers/ubi10/ubi-minimal/66f1504a379b9c2cf23e145c#get-this-image
5+
# IMPORTANT: Make sure to use the "Manifest List Digest" that references the images for multiple architectures
6+
# rather than just the "Image Digest" that references the image for the selected architecture.
7+
FROM registry.access.redhat.com/ubi10/ubi-minimal@sha256:28ec2f4662bdc4b0d4893ef0d8aebf36a5165dfb1d1dc9f46319bd8a03ed3365
98

109
ARG PRODUCT_VERSION
10+
ARG PYTHON_VERSION
1111
ARG RELEASE_VERSION
12-
ARG KEYCLOAK_VERSION
1312
ARG STACKABLE_USER_UID
1413
ARG STACKABLE_USER_GID
1514
ARG STACKABLE_USER_NAME
@@ -25,50 +24,54 @@ LABEL name="Stackable Testing Tools" \
2524
# https://github.com/hadolint/hadolint/wiki/DL4006
2625
SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
2726

28-
# This is needed so that krb5-user installs without prompting for a realm.
29-
ENV DEBIAN_FRONTEND=noninteractive
3027

28+
# We configure microdnf to not install weak dependencies in this file
29+
# Not doing this caused the content of images to become unpredictable because
30+
# based on which packages get updated by `microdnf update` new weak dependencies
31+
# might be installed that were not present earlier (the ubi base image doesn't
32+
# seem to install weak dependencies)
33+
# This also affects the packages that are installed in our Dockerfiles (java as prime
34+
# example).
35+
# https://github.com/stackabletech/docker-images/pull/533
36+
COPY stackable-base/stackable/dnf.conf /etc/dnf/dnf.conf
3137

32-
COPY testing-tools/python /stackable/python
33-
COPY testing-tools/licenses /licenses
38+
# Default curl configuration to avoid forgetting settings and to declutter the Dockerfiles
39+
COPY stackable-base/stackable/curlrc /root/.curlrc
3440

41+
# Base requirements for all testing-tools images
42+
COPY testing-tools/requirements.txt /stackable/requirements.txt
43+
COPY testing-tools/licenses /licenses
3544

3645
RUN <<EOF
37-
apt-get update
38-
apt-get install -y --no-install-recommends \
39-
build-essential \
40-
ca-certificates \
41-
curl \
42-
gzip \
43-
jq \
44-
`# krb5-user/libkrb5-dev are needed for Kerberos support. ` \
45-
krb5-user \
46-
libkrb5-dev \
47-
kubernetes-client \
48-
libssl-dev \
49-
libxml2-dev \
50-
libxslt1-dev \
46+
microdnf update
47+
48+
microdnf install \
49+
gcc \
50+
make \
5151
pkg-config \
52-
python3-certifi \
53-
python3-idna \
54-
python3-semver \
55-
python3-thrift \
56-
python3-toml \
57-
python3-urllib3 \
52+
openssl-devel \
53+
libxml2-devel \
54+
libxslt-devel \
55+
python${PYTHON_VERSION} \
56+
python${PYTHON_VERSION}-devel \
57+
python${PYTHON_VERSION}-pip \
58+
jq \
59+
gzip \
60+
curl \
5861
tar \
5962
zip \
60-
unzip \
61-
`# Java 11 seems like the best middle-ground for all tools` \
62-
openjdk-11-jdk-headless
63+
unzip
6364

64-
apt-get clean
65-
rm -rf /var/lib/apt/lists/*
65+
microdnf clean all
66+
rm -rf /var/cache/yum
6667

67-
curl --fail -L https://repo.stackable.tech/repository/packages/keycloak/keycloak-${KEYCLOAK_VERSION}.tar.gz | tar -xzC /stackable
68-
ln -s /stackable/keycloak-${KEYCLOAK_VERSION} /stackable/keycloak
68+
python3.12 -m pip install --no-cache-dir --upgrade pip
69+
python3.12 -m pip install --no-cache-dir -r /stackable/requirements.txt
6970

70-
pip install --no-cache-dir --upgrade pip
71-
pip install --no-cache-dir -r /stackable/python/requirements.txt
71+
ln -s /usr/bin/python3.12 /usr/bin/python
72+
73+
# Added only temporarily to create the user and group, removed again below
74+
microdnf install shadow-utils
7275

7376
groupadd --gid ${STACKABLE_USER_GID} --system ${STACKABLE_USER_NAME}
7477
useradd \
@@ -80,11 +83,12 @@ useradd \
8083
--home-dir /stackable \
8184
${STACKABLE_USER_NAME}
8285

86+
microdnf remove shadow-utils
87+
microdnf clean all
88+
8389
chown -R ${STACKABLE_USER_UID}:0 /stackable
8490
EOF
8591

86-
ENV PATH=/stackable/keycloak/bin:$PATH
87-
8892
USER ${STACKABLE_USER_UID}
8993

9094
ENV STACKABLE_PRODUCT_VERSION=${PRODUCT_VERSION}

‎testing-tools/boil-config.toml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
[versions."0.2.0".build-arguments]
2-
keycloak-version = "26.3.5"
1+
[versions."0.3.0".build-arguments]
2+
python-version = "3.12"

‎testing-tools/hive/Dockerfile‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# syntax=docker/dockerfile:1.16.0@sha256:e2dd261f92e4b763d789984f6eab84be66ab4f5f08052316d8eb8f173593acf7
2+
# check=error=true
3+
4+
FROM local-image/testing-tools
5+
6+
ARG PRODUCT_VERSION
7+
ARG RELEASE_VERSION
8+
ARG STACKABLE_USER_UID
9+
10+
LABEL name="Stackable Testing Tools - Hive" \
11+
maintainer="info@stackable.tech" \
12+
vendor="Stackable GmbH" \
13+
version="${PRODUCT_VERSION}" \
14+
release="${RELEASE_VERSION}" \
15+
summary="Stackable tools for Hive integration tests." \
16+
description="Hive-specific testing tools image based on testing-tools base image."
17+
18+
# https://github.com/hadolint/hadolint/wiki/DL4006
19+
SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
20+
21+
USER root
22+
23+
COPY testing-tools/hive/requirements.txt /stackable/hive/requirements.txt
24+
25+
RUN <<EOF
26+
microdnf update
27+
28+
microdnf install \
29+
krb5-workstation \
30+
krb5-devel
31+
32+
microdnf clean all
33+
rm -rf /var/cache/yum
34+
35+
pip install --no-cache-dir -r /stackable/hive/requirements.txt
36+
EOF
37+
38+
USER ${STACKABLE_USER_UID}
39+
40+
ENV STACKABLE_PRODUCT_VERSION=${PRODUCT_VERSION}
41+
42+
WORKDIR /stackable

‎testing-tools/hive/boil-config.toml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[versions."0.3.0".local-images]
2+
testing-tools = "0.3.0"

‎testing-tools/hive/requirements.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
hive-metastore-client==1.0.9
2+
thrift==0.13.0
3+
4+
# gssapi/kerberos/pure-sasl are needed for using the python metastore client with kerberos.
5+
gssapi==1.10.1
6+
kerberos==1.3.1
7+
pure-sasl==0.6.2

0 commit comments

Comments
 (0)