[CDRIVER-6142] Refactor dependency/environment setup in Earthfile #2165
+2,194
−1,021
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change rewrites much of Earthfile's dependency-installation handling code to be simpler to modify, and more flexible to use. Previously, each environment had its own target that needed to know how to install all of our dependencies for the specific platform. This refactor inverts the relationship: Instead, each dependency has an Earthly function that knows how to install itself based on whatever platform it happens to be running within. This allows the FROM line to be a plain image, rather that one of our handcrafted `env.foo` targets. This will make introducing and testing additional environments far more simple in the future. This change also allows us to remove the tools/build.earth utility, since we can now fit everything in the top-level file.
vector-of-bool
requested review from
connorsmacd and
eramongodb
and removed request for
connorsmacd
Contributor
Author
|
@eramongodb requested you especially because this does some juggling of the ECR handling in EVG. |
eramongodb
requested changes
Nov 10, 2025
Contributor
eramongodb
left a comment
eramongodb
left a comment
There was a problem hiding this comment.
The Earthly workflow is a lot easier to track than before. Thank you for these improvements!
eramongodb
approved these changes
Nov 13, 2025
Contributor
eramongodb
left a comment
eramongodb
left a comment
There was a problem hiding this comment.
Minor suggestions/questions remaining; otherwise, LGTM.
Comment on lines
+406
to
+408
| # devdocs : | ||
| # Builds the developer documentation pages as HTML, and writes the resulting pages into | ||
| # `_build/devdocs` on the host for browsing. |
| ARG uv_version = "0.8.15" | ||
| ARG uv_install_sh_url = "https://astral.sh/uv/$uv_version/install.sh" | ||
| IF ! __have_command uv | ||
| RUN curl -LsSf "$uv_install_sh_url" \ |
Contributor
There was a problem hiding this comment.
Even if not for the source tarballs downloaded by the install script, recommend adding checksum validation for the install script itself.
| # shellcheck disable=SC3043 | ||
|
|
||
| true <<EOF | ||
| This script is inteded for use with preparing a container environment, providing |
Contributor
There was a problem hiding this comment.
Suggested change
| This script is inteded for use with preparing a container environment, providing | |
| This script is intended for use with preparing a container environment, providing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This changeset modifies how Earthly environments are specified and set up for the build.
While doing refactors to the
amongocCI plan, I found that it is far simpler to have an Earthly function for each dependency that knows how to "do the right thing" to install the relevant packages for whatever host platform it happens to be running within, especially by adding a new__can_installutility. The following significant changes are made:env.fooEarthly targets are dropped. Instead, environments are specified using--frominstead of--env, which just executes a regularFROMcommand to set the base container image.ADD_FOOEarthly function is defined, and it will just "do the right thing" to obtain the relevant package for the build (or do nothing, if that dependency is disabled).__toolscript, which explodes itself using__aliasto reproduce the same set of tools that were available beforehand.__can_install <pkg>, which determines whether a package<pkg>is available for installation. This makes dep management far simpler since we don't need to do fancy platform/tool detection when we can just say "Ifgcc-c++is available, install it." without needing to know which package manager we're going to use.--from(formerly--env) has been modified to take a plain container image specifier, the logic for switching to ECR for the base platform container images is moved toearthly.py.+init,+build-environment,+configure, and+build. The main purpose of this split is to support subdividing the EVG build into the four steps so that we get separate timing/logging information for each.All of the above also has the added benefit of being able to test any Linux distro/version on-the-fly by just passing a new
--fromrather than needing to modifyEarthfileto support a new platform.