Add a JSON Schema for `Derivation` #14269

roberth · 2025-10-16T13:36:22Z

Motivation

Let's be serious about JSON.

Schema
Docs generation from the schema
Validation checks

Adds a python tool to the manual build.
Example validation is a separate check; not part of the nix package. Runs in CI.

Context

Based loosely on what nixops4 does with its JSON (except we don't generate code; different but orthogonal question)

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

Ericson2314 · 2025-10-16T19:18:44Z

OK this now goes to master, and just does existing serializations. https://github.com/nixos/nix/tree/json-schema-hash has the hash changes (no they aren't lost!) can land that next.

For manual, and testing formats

Ericson2314 · 2025-10-16T21:30:53Z

@roberth if you informally approve the parts that I did, I am happy with the parts that you did, and then we can merge it.

Note that I changed it around so that the tests data's canonical location is back in the data dir. I am a bit fond of that in general, but also I think it will make making new tests easier because _NIX_TEST_ACCEPT=1 will put them in the right spot, vs needing to move them afterwards.

Ericson2314 · 2025-10-16T21:31:56Z

src/json-schema-checks/meson.build

+  # # Not sure how to make subschema work
+  # {
+  #   'stem': 'derivation',
+  #   'schema': schema_dir / 'derivation-v3.yaml#output',
+  #   'files' : [
+  #     'output-caFixedFlat.json',
+  #     'output-caFixedNAR.json',
+  #     'output-caFixedText.json',
+  #     'output-caFloating.json',
+  #     'output-deferred.json',
+  #     'output-impure.json',
+  #     'output-inputAddressed.json',
+  #   ],
+  # },


I think this is fine for now. If we can't get a better solution / get an upstream fix to jv, when I later improve the derivation output schema, I will just move it to separate file(s) so this works.

Ericson2314 · 2025-10-16T21:32:56Z

src/nix/derivation-add.md

+`nix derivation add` takes a single derivation in the JSON format.
+See [the manual](@docroot@/protocols/json/derivation.md) for a documentation of this format.


I figured whatever markdown was being generated looked pretty fancy, and it is probably too fancy for the man page.

Ericson2314 · 2025-10-16T21:33:11Z

src/nix/derivation-show.md

-{{#include ../../protocols/json/derivation.md}}
-


(Same as above)

Ericson2314 · 2025-10-16T21:34:08Z

doc/manual/source/protocols/json/fixup-json-schema-generated-doc.sed

+# For some reason, backticks in the JSON schema are being escaped rather
+# than being kept as intentional code spans. This removes all backtick
+# escaping, which is an ugly solution, but one that is fine, because we
+# are not using backticks for any other purpose.
+s/\\`/`/g


We should make an upstream issue about this, but that can happen later

Ericson2314 · 2025-10-16T21:34:54Z

doc/manual/source/protocols/json/schema/derivation-v3.yaml

+      > **Example**
+      >
+      > ```json
+      > "outputs": {
+      >   "out": {
+      >     "method": "nar",
+      >     "hashAlgo": "sha256",
+      >     "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
+      >   }
+      > }
+      > ```


JSON Schema is supposed to support examples, but they were not rendering so I just did this instead.

Ericson2314 · 2025-10-16T21:35:58Z

doc/manual/source/protocols/json/schema/derivation-v3.yaml

+  output:
+    type: object
+    properties:
+      path:
+        type: string
+        description: |
+          The output path, if known in advance.
+
+      method:
+        type: string
+        enum: [flat, nar, text, git]
+        description: |
+          For an output which will be [content addressed](@docroot@/store/derivation/outputs/content-address.md), a string representing the [method](@docroot@/store/store-object/content-address.md) of content addressing that is chosen.
+
+          Valid method strings are:
+
+          - [`flat`](@docroot@/store/store-object/content-address.md#method-flat)
+          - [`nar`](@docroot@/store/store-object/content-address.md#method-nix-archive)
+          - [`text`](@docroot@/store/store-object/content-address.md#method-text)
+          - [`git`](@docroot@/store/store-object/content-address.md#method-git)
+
+      hashAlgo:
+        "$ref": "./hash-v1.yaml#/$defs/algorithm"
+
+      hash:
+        type: string
+        description: |
+          For fixed-output derivations, the expected content hash in base-16.


We could do a better sum type than this for sure, but I just kept it like this for now because it matches how the old manually-written docs were.

Ericson2314 · 2025-10-16T21:37:14Z

doc/manual/source/protocols/json/json-schema-for-humans-config.yaml

+with_footer: false
+recursive_detection_depth: 3
+show_breadcrumbs: false
+description_is_markdown: true


I was hoping this would fix the backtick issue, but it did not.

Figured it out: only happens when the description gets used in place of our missing title.
By providing titles we get a brief ToC in the table, and the rest of the content in its own section, without formatting messes.

I've also opened coveooss/json-schema-for-humans#319 to make that relation impossible to miss.

Ericson2314 · 2025-10-16T21:38:03Z

doc/manual/source/protocols/json/hash.md

+<!--
+## Raw Schema
+
+[JSON Schema for Hash v1](schema/hash-v1.json)
+-->


It would be nice to generate the raw JSON from the YAML, but I didn't do that yet. I think it can wait.

(We could also go back to JSON from YAML for the source files, but I think that is a bit bad for git diffs and multi-line strings.

roberth · 2025-10-16T21:08:51Z

doc/manual/source/protocols/json/schema/derivation-v3.yaml

+    type: object
+    description: |
+     Information about the output paths of the derivation.
+     This is a JSON object with one member per output, where the key is the output name and the value is a JSON object as described.


We could borrow the technical term from the JSON guidelines,

Suggested change

This is a JSON object with one member per output, where the key is the output name and the value is a JSON object as described.

This JSON object is a dictionary with one member per output, where the key is the output name and the value is a JSON object as described.

or use a less explicit but more natural term:

Suggested change

This is a JSON object with one member per output, where the key is the output name and the value is a JSON object as described.

This JSON object is a mapping with one member per output, where the key is the output name and the value is a JSON object as described.

roberth · 2025-10-17T19:37:02Z

doc/manual/source/protocols/json/json-schema-for-humans-config.yaml

+with_footer: false
+recursive_detection_depth: 3
+show_breadcrumbs: false
+description_is_markdown: true


Figured it out: only happens when the description gets used in place of our missing title.
By providing titles we get a brief ToC in the table, and the rest of the content in its own section, without formatting messes.

I've also opened coveooss/json-schema-for-humans#319 to make that relation impossible to miss.

This way, the description isn't rendered in the tables of contents, leading to no more formatting errors.

roberth · 2025-10-18T18:50:03Z

Since we both worked on this, your contributions (and I think mine) LGTM!

roberth requested a review from edolstra as a code owner October 16, 2025 13:36

github-actions bot added the documentation label Oct 16, 2025

roberth mentioned this pull request Oct 16, 2025

JSON impl and Schema for DummyStore #13942

Merged

roberth force-pushed the json-schema branch from 4218451 to 7c58407 Compare October 16, 2025 13:38

Ericson2314 force-pushed the json-schema branch from 7c58407 to 6839d7f Compare October 16, 2025 19:04

Ericson2314 self-requested a review as a code owner October 16, 2025 19:04

github-actions bot added the new-cli Relating to the "nix" command label Oct 16, 2025

Ericson2314 changed the title ~~Add a JSON Schema for Hash~~ Add a JSON Schema for Derivation Oct 16, 2025

Ericson2314 force-pushed the json-schema branch 2 times, most recently from 1475a77 to f5e6a86 Compare October 16, 2025 19:15

Ericson2314 changed the base branch from json-no-store-dir to master October 16, 2025 19:17

Ericson2314 force-pushed the json-schema branch from f5e6a86 to 3a851a7 Compare October 16, 2025 20:11

Add JSON Schema infrastructure, use for Derivation

01b001d

For manual, and testing formats

Ericson2314 force-pushed the json-schema branch from 3a851a7 to 01b001d Compare October 16, 2025 21:24

Ericson2314 approved these changes Oct 16, 2025

View reviewed changes

Ericson2314 reviewed Oct 16, 2025

View reviewed changes

src/nix/derivation-show.md

Comment on lines -55 to -56

{{#include ../../protocols/json/derivation.md}}

Copy link

Member

Ericson2314 Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Same as above)

Ericson2314 reviewed Oct 16, 2025

View reviewed changes

roberth mentioned this pull request Oct 17, 2025

feat: Add (read more) links to md format coveooss/json-schema-for-humans#319

Open

roberth commented Oct 17, 2025

View reviewed changes

Add titles in JSON schemas

c92ba4b

This way, the description isn't rendered in the tables of contents, leading to no more formatting errors.

Ericson2314 added this pull request to the merge queue Oct 18, 2025

Merged via the queue into NixOS:master with commit b56e456 Oct 18, 2025
16 checks passed

		`nix derivation add` takes a single derivation in the JSON format.
		See [the manual](@docroot@/protocols/json/derivation.md) for a documentation of this format.

	This is a JSON object with one member per output, where the key is the output name and the value is a JSON object as described.
	This JSON object is a dictionary with one member per output, where the key is the output name and the value is a JSON object as described.

	This is a JSON object with one member per output, where the key is the output name and the value is a JSON object as described.
	This JSON object is a mapping with one member per output, where the key is the output name and the value is a JSON object as described.

Uh oh!

Add a JSON Schema for Derivation #14269

Add a JSON Schema for Derivation #14269

Uh oh!

Conversation

roberth commented Oct 16, 2025 • edited by Ericson2314 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Context

Uh oh!

Ericson2314 commented Oct 16, 2025

Uh oh!

Ericson2314 commented Oct 16, 2025

Uh oh!

Ericson2314 Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

roberth commented Oct 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add a JSON Schema for `Derivation` #14269

Add a JSON Schema for `Derivation` #14269

roberth commented Oct 16, 2025 •

edited by Ericson2314

Loading

Ericson2314 Oct 16, 2025 •

edited

Loading