Should the default `properties.schema` version be changed to the recommended 2.0?

[joelostblom]

The Data Package docs state the following for properties.schema:

The default value is https://datapackage.org/profiles/1.0/datapackage.json and the recommended value is https://datapackage.org/profiles/2.0/datapackage.json

The value in the schema used in this repo is 1.0:

check-datapackage/src/check_datapackage/schemas/data-package-2-0.json

Line 11 in fc67bf8

"default": "https://datapackage.org/profiles/1.0/datapackage.json",

Should we change it to 2.0?

I will say that am not sure when this default schema is used; maybe there is a reason it was kept at 1.0 but I thought I would raise this in case it was just missed when copying over the schema?

[lwjohnst86]

We take directly from the data package schema, and in their schema, it is 1.0. I'm not sure if we should modify the schema unless they modify it. We copy their schema exactly, no modifications.

[joelostblom]

The quote above is from their docs where they recommend it to be 2.0. Maybe the default of 1.0 is there for some legacy reason that and there are advantages of changes to 2.0, which is why the recommend everyone to switch? Do you know what this value is even for, is it some sort of fallback?

[lwjohnst86]

I have noooo idea 🤷

[joelostblom]

Pinging @seedcase-project/developers in case anyone else knows. If not, I can ask on their issue tracker as I didn't understand it from the docs myself.

[martonvago]

Oh, I know something about this, one sec!

[martonvago]

I've also noticed this and my best guess is that they kept the default as v1.0 because most of their users are still using v1.0, and changing the default to v2.0 would mess up their data package validation (using the frictionless-py package).

The way I understand it, $schema allows you to declare which profile your descriptor is compliant with. When they say the recommended value is 2.0, they are saying that in your datapackage.json that defines your Data Package (so not the JSON schema), $.$schema should be set to 2.0. If you don't set it to 2.0, it will default to 1.0. It defaulting to 1.0 is defined in the JSON schema (the line you linked).

We currently only support checking against the 2.0 schema, but I don't think this is flagged explicitly to the user. If someone called check on a descriptor that was v1.0 compliant (but not v2.0 compliant), I guess various checks would fail, but we wouldn't tell the user that this is because we were checking against v2.0.

So now it seems to me that the correct thing to do is:

• check the $schema property in the input descriptor and raise an error if it's not v2.0
• in our example descriptors (and any descriptors created in Sprout) we should set the $schema property to v2.0

[joelostblom]

Ah great, thanks for the explanation @martonvago! I think the two actions you suggested seem appropriate 👍 Just one question on the first one:

check the $schema property in the input descriptor and raise an error if it's not v2.0

It makes sense to me to raise and error if the specified version is <2, but I'm unsure what to do if it is left undefined (if this is allowed). Should we still raise an error or default to check their schema against v2 (by modifying our default schema to v2 in our data-package-2-0.json as per the recommendation)? If the latter, we could emit a warning to make it explicit that since the version was undefined in their datapackage.json we defaulted to checking against v2.

[martonvago]

$schema is allowed to be undefined, this is when the default kicks in. So if we left the schema as is, an undefined $schema would be interpreted as v1 and therefore rejected in check-datapackage. To be clear, there is no code that does this currently, I just mean logically.

Seeing that both check-datapackage and Sprout are meant to work with v2 Data Packages (primarily?), I can see the argument for changing our default to v2. Precisely so that an undefined $schema doesn't lead to an error.

Now, changing our default would just mean not throwing an error if $schema is undefined. That is to say, we never actually read the default $schema value in the JSON schema and make decisions based on it. So we could change our default without changing the default in the schema. But then our implementation would contradict the schema.

We could change the default in the schema as well, but then it wouldn't be the Data Package schema exactly. I don't think there are practical problems with this, but I don't know if we want to deviate from their schema at all. Maybe @lwjohnst86 has a view on that.

(Also, just to make sure it's clear, the v2 recommendation applies to the descriptor, not the schema. They are not suggesting that the default value in the schema should be changed.)