feat(mongodb-runner): make suitable for drivers team use DRIVERS-3335 #598

addaleax · 2025-11-18T02:54:57Z

This adds new features to the mongodb-runner package, both for programmatic and CLI usage, in order to let drivers adopt this as their main MongoDB cluster management tool.

CLI improvements:

Support --config <file> to provide a config file
Let users trigger verbose debug mode through the CLI
Let users specify config options via environment variables

Core features:

Custom arguments for the individual replset members, individual shards, or individual mongoses (and multiple mongoses, through that)
Support for enabling authentication on clusters as part of their startup

Related: mongodb-labs/drivers-evergreen-tools#701

…ectly

packages/mongodb-runner/src/mongocluster.spec.ts

Anemy · 2025-11-18T04:09:18Z

packages/mongodb-runner/src/mongocluster.ts

+  async addAuthIfNeeded(): Promise<void> {
+    if (!this.users?.length) return;
+    // Sleep to give time for a possible replset election to settle.
+    await sleep(1000);


Is there something that could tell us a replset election is done earlier? Seems like a place where things could add up timewise.

Yeah, this is taken straight from @blink1073's commits, I've asked myself the same question but he can probably answer more easily. We do wait for a primary to be self-reporting as the winner of the election before, so I'm not entirely sure that this is necessary

Despite the wait for the self-report, I was getting a notPrimary error about 50% of the time before I added the sleep.

Another wrinkle: if a keyFile is used, the initial metadata insert fails with the error below. In my branch I deferred the buildInfo check if there was a keyFile. The cluster start still works in the end with the failure.

DEBUG: mongodb-runner failed to get buildInfo, treating as closed server MongoServerError: Authentication failed. at Connection.sendCommand (/Users/steve.silvester/workspace/drivers-evergreen-tools/.evergreen/orchestration/node_modules/mongodb/lib/cmap/connection.js:306:27) at process.processTicksAndRejections (node:internal/process/task_queues:103:5) at async Connection.command (/Users/steve.silvester/workspace/drivers-evergreen-tools/.evergreen/orchestration/node_modules/mongodb/lib/cmap/connection.js:334:26) at async executeScram (/Users/steve.silvester/workspace/drivers-evergreen-tools/.evergreen/orchestration/node_modules/mongodb/lib/cmap/auth/scram.js:79:22) at async ScramSHA256.auth (/Users/steve.silvester/workspace/drivers-evergreen-tools/.evergreen/orchestration/node_modules/mongodb/lib/cmap/auth/scram.js:39:16) at async performInitialHandshake (/Users/steve.silvester/workspace/drivers-evergreen-tools/.evergreen/orchestration/node_modules/mongodb/lib/cmap/connect.js:104:13) at async connect (/Users/steve.silvester/workspace/drivers-evergreen-tools/.evergreen/orchestration/node_modules/mongodb/lib/cmap/connect.js:24:9) { errorLabelSet: Set(2) { 'HandshakeError', 'ResetPool' }, ...

Anemy · 2025-11-18T04:10:46Z

packages/mongodb-runner/src/mongocluster.ts

  isClosed(): boolean {
-    return this.servers.length === 0 && this.shards.length === 0;
+    // eslint-disable-next-line @typescript-eslint/no-unused-vars
+    for (const _ of this.children()) return true;


Should it be returning false if there are children? Not sure I follow the logic here, maybe worth a comment? I might be missing something.

This matches the existing logic – if all servers have been removed from the cluster, than it's closed. But I'm happy to add a comment here

Should it be

Suggested change

for (const _ of this.children()) return true;

for (const _ of this.children()) return false;

Then? Wouldn't return true mean there are servers or shards? Thanks for adding a comment! Apologies if I'm totally missing this.

blink1073 · 2025-11-18T12:52:36Z

packages/mongodb-runner/src/cli.ts

      type: 'string',
      describe: 'Configure OIDC authentication on the server',
    })
+    .config()


Haha I can't believe I missed this option. 😅

I verified that it passes options through that aren't defined in the cli options, so we don't need to add all of the new options to the cli.

packages/mongodb-runner/src/mongocluster.ts

blink1073 · 2025-11-18T15:45:18Z

packages/mongodb-runner/src/mongocluster.ts

+      rsMembers: RSMemberOptions[];
+    };
+
+export type ShardedOptions = {


The way the shards are defined in our config files is more like a list of rsMembers, so I think we need to add:

| { shards?: never; shardArgs?: RSMemberOptions[][]; } ```

Yep, will do

blink1073 · 2025-11-18T16:36:35Z

packages/mongodb-runner/src/mongocluster.ts

+
+export type MongoClusterOptions = Pick<
+  MongoServerOptions,
+  'logDir' | 'tmpDir' | 'args' | 'binDir' | 'docker'


We need to pick defaultConnectionOptions here as well to handle the TLS options.

That doesn't sound quite right – can you describe the use case here?

How else will internal client returned by withClient be able to use TLS?

We pass these options on to mongo-orchestration to use in its connection string for internal clients:

ssl = true tlsCertificateKeyFile = clientCertFile tlsAllowInvalidCertificates = true

@blink1073 Sooo ... I've pushed updates here, and yes, this will let you override flags for the internal client, but I'd generally like to keep usage of this feature minimal.

What I've done here is a bit more complex: By default, when mongodb-runner detects TLS usage, it creates a TLS client key and self-signed certificate on the fly, and adds those to the server's CA file. That's a bit more in line with the philosophy of "one toggle does one thing" that we try to stick to here in the package's interface, i.e. you will generally just need to specify --tlsCAFile on the command line and the rest of it will be handled for you.

Okay that sounds reasonable to me, thank you! To keep this PR scoped, I think we should stop here feature-wise. Some ideas for follow up:

Better handling of keyFile and its interaction with the internal metadata.

Allow args to be passed as part of the config file and appended to the command line args

Add support for unix sockets

Write to stderr for all except the connection string, so a calling process can capture it directly.

I'll test out these new changes and get back to you!

Yeah, for keyfiles I think we'll want to do something similar to TLS – intercept those and use it for auth, if possible.

But yeah, let's just try to get this work here over the line 🙂

Everything is working now except for the shards config option

blink1073 · 2025-11-25T12:04:27Z

packages/mongodb-runner/src/cli.ts

      describe: 'Configure OIDC authentication on the server',
    })
+    .config()
+    .env('MONGODB_RUNNER')


Since "shards" can be an object now, I think shards needs a custom coerce method. Right now if it is an object it gets converted to NaN.

Note: I'm testing this using the cli instead of programmatically, to align more closely with mongotune and to avoid adding a node project to the orchestration folder (to minimize overall change).

Here's that new PR: mongodb-labs/drivers-evergreen-tools#706

blink1073 and others added 14 commits November 18, 2025 03:48

Add additional options to MongoCluster

a317cce

wip add auth support

b6beb60

finish support for auth and ssl

bde597f

finish sharded support

462f151

align more with mongodb-runner design principles

28a10e2

fix handling of arbiters

5e383ec

Add RSMemberOptions

e475696

fixup: use actual TagSet type

625c540

feat(mongodb-runner): add ability to use a config file

47de6fd

fixup: clean up replset and shard arg handling

4c756a1

fixup: allow passing CLI args as env vars, add verbose debug mode dir…

23bb0d3

…ectly

fixup: be more correct/more explicit in replset hosts test

d159b3d

add tests, fix behaviors

fea6ceb

fixup: check mongos list in config db

74b8c93

Anemy reviewed Nov 18, 2025

View reviewed changes

fixup: CR, use 8.x in tests by default

008bfdf

blink1073 reviewed Nov 18, 2025

View reviewed changes

fixup: version assertions

04249ed

blink1073 reviewed Nov 18, 2025

View reviewed changes

packages/mongodb-runner/src/mongocluster.ts Show resolved Hide resolved

fixup: cr, export interfaces, fix auth test for 8.x

4746fa0

blink1073 reviewed Nov 18, 2025

View reviewed changes

addaleax added 3 commits November 24, 2025 23:29

fixup: support TLS on internal client out of the box

acfe911

fixup: full flexibility for individual shards

401163c

chore: add documentation

42cc373

addaleax requested a review from blink1073 November 24, 2025 22:59

addaleax added 2 commits November 25, 2025 00:02

fixup: bump to Node.js 22.x in CI

d919fe9

fixup: update tests

41b1b32

blink1073 reviewed Nov 25, 2025

View reviewed changes

	for (const _ of this.children()) return true;
	for (const _ of this.children()) return false;

feat(mongodb-runner): make suitable for drivers team use DRIVERS-3335 #598

Are you sure you want to change the base?

feat(mongodb-runner): make suitable for drivers team use DRIVERS-3335 #598

Uh oh!

Conversation

addaleax commented Nov 18, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Anemy Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

blink1073 Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

blink1073 Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

blink1073 Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Anemy Nov 18, 2025 •

edited

Loading

blink1073 Nov 18, 2025 •

edited

Loading

blink1073 Nov 18, 2025 •

edited

Loading

blink1073 Nov 25, 2025 •

edited

Loading