Support JSON Schema Draft 2020-12

[magicmatatjahu]

Currently in the 2.1.0 version of spec, we support only JSON Schema in the version of draft 07. There are some changes (including breaking changes) between draft 07 -> draft 2019-09 -> draft 2020-12, especially with core keywords:

• dependencies are split to dependentSchemas and dependentRequired - as I read dependencies has still backward compatibility
• added deprecated keyword - similar to our deprecated
• added new predefined formats
• added unevaluatedItems and unevaluatedProperties keywords - similar to additional*, but they can "see" into subschemas and across references
• added local/dynamic anchors
• added minContains, maxContains
• definitions -> $def (definitions has still backward compatibility)
• in the $ref other keywords are now allowed alongside of it
• ... other thing related to the vocabularies and meta schemas

You can read all changes in:

• Draft-07 to Draft 2019-09
• Draft 2019-09 to Draft 2020-12

Due to the fact that JSON Schema itself has breaking change and it's treated by us as a base for our Schema Object, so the specification itself will have breaking change when we start supporting the latest drafts.

The issue itself is like question: When will we start supporting this? O maybe someone have an opinion that we shouldn't support this at all in the near future? Or if yes, when? In 3.0.0 or 4.0.0?

We must also remember about adjusting the JSON Schema for spec to the Draft 2020-12.

[Relequestual]

A thought:
If you support 2019-09 or 2020-12, you can also still support draft-07 (and as many versions as you want to, via dialects).
It's just you define the dialect you're using with $schema.
Supporting multiple versions of JSON Schema at the same time MAY seem problematic, but it would give you backwards compatibility, making it a backwards compatible change as opposed to a breaking change.

[fmvilas]

@Relequestual I think we'll need your help with dialects 😊 Mind joining our next SIG meeting and giving us an introduction/demo?

[Relequestual]

@fmvilas Sorry for not seeing this. I rarely get round to seeing GitHub notifications.
"Dialects" are essentially collections of vocablaries.
Each version of JSON Schema is a "dialect". If you look at the 2020-12 meta-schema, you'll see it's comprised of multiple vocabularies.

When processing a JSON Schema, an implementation must look at any defined $schema values to determine the dialects used. Usually this is only one dialect in the root of the document.

In order to determine if you can process a document, you must know the defined dialect, and be able to match that to a meta-schema.
The meta-schema defines the vocabulary requirements for that dialect.
If you match the dialect URI to a meta-schema, you have an error, because you cannot proceed.
If you can, you must then know the vocabularies that are required for correct processing defined in the meta-schema.

Let me know if any of that doesn't make sense.
(Then let me know you've replied in slack 😅)

[magicmatatjahu]

@Relequestual Yeah, I understand that, user can choose which version of JSON Schema can use by $schema, but we have "hardcoded" in our JSON Schema for spec that user can only use draft-07, so this issue is related to that, to support by default latest version, but if someone will want to use older, then should use $schema :)

[Relequestual]

Great! Just wanted to share as @fmvilas said you needed help with dialects. I guess that's outdated?

[smoya]

@Relequestual Could we use 2019-09 (or greater) and add all required vocabularies from Draft-07 so we can keep retro compatibility?
The point is that Draft-07 doesn't have any defined vocabulary (because vocabularies were introduced in 2019-09) so I don't see how we could use it, unless we manually host new vocabulary for those Draft-07 vocabularies we wanna keep then use them through something like:

"allOf": [                                                             
    { 
        "$ref": "https://asyncapi.com/220-schema",
        "$ref": "https://asyncapi.com/json-schemadraft-7-vocabulary",
    }
  ],

[Relequestual]

@smoya You can use any "dialect" that is known.
Prior to 2019-09, for example draft-07, the dialect is a single vocabulary, but never defined.
As such, you could argue the dialect and the vocaublary URI are the same.
(Although, this could cause problems if we later decide that the vocabulary URI as a URL can provide some data.)

Are you extending the draft-07 dialect/vocabulary with your own keywords?
If so, it sounds like you want JSON Schema to define a vocabulary URI for draft-07 (and previous), so you can use it along with your own vocabularies to define your own dialect. Does that sound right?

If I understand correctly, we could look into providing those vocabulary URIs, but how far back should we go? draft-04? We want to encourage people to move to newer versions.

[smoya]

There is a potential blocker that may stop us to move forward from Draft-07 in terms of tooling. json-schema-ref-parser, the tool used for parsing JSON Reference and JSON Pointer withing our Parser-JS, does not support JSON Schema 2019-09 or greater yet. See APIDevTools/json-schema-ref-parser#145.
Dependant tooling such as JSON Schema types are not available for versions greater than Draft-07

Meaning some of $ref features won't work if we move forward.

We should either look for an alternative, work on such linked issue or craft a solution ourselves.

EDIT: Related issue: asyncapi/parser-js#404

[ThanksForAllTheFish]

I was chatting with @jonaslagoni in slack and he suggested me to leave a comment here. I would like to see the upgrade as well, not sure I can contribute much practically (don't know json schema that deeply, don't know js much either), so consider this more like an end user desire, but draft 7 is pretty old now and new people (like me) to json schema would probably start with newer drafts. Also, I think OpenAPI 3.1 is backed by json schema 2020, so it would make sense to upgrade there

[fmvilas]

My only concern is the state of the tooling support for the newer versions of JSON Schema. The last time I checked, it wasn't widely supported. If tooling is not there yet, I'd not upgrade either. There's no point in upgrading to the latest version if fewer people will be able to use it.

[smoya]

My only concern is the state of the tooling support for the newer versions of JSON Schema. The last time I checked, it wasn't widely supported. If tooling is not there yet, I'd not upgrade either. There's no point in upgrading to the latest version if fewer people will be able to use it.

If we stay, there are still some features from JSON Schema Draft 2019-09 and greater that we could port, such as the reference and bundling behavior. Tooling would need to be maintained in our side but maybe it is worth the movement. More on #649

[fmvilas]

Tooling would need to be maintained in our side but maybe it is worth the movement.

I honestly doubt it but I don't have the data, it's just a gut feeling. I think most people are not using complex JSON Schemas on AsyncAPI. But again, just a gut feeling.

[smoya]

I honestly doubt it but I don't have the data, it's just a gut feeling. I think most people are not using complex JSON Schemas on AsyncAPI. But again, just a gut feeling.

Yeah, I think so as well. In fact, the only needed (at a glance) feature was having the possibility to override properties when using $ref, but after latest findings I think the behavior JSON Schema does starting from Draft 2019-09 won't suit our use cases.

For example, and this is the most unsuitable thing IMHO, is the fact that properties to override will do a AND with the referenced schema. I.e.:

{"type": "object", "$ref": "foo"} is equivalent to {"type": "object", "allOf": [{... contents of foo schema ...}]}

We (@jonaslagoni and I) couldn't find an official doc place where they mention this, however you can find some mentions on this library issue APIDevTools/json-schema-ref-parser#145 (comment) (from where I got the example above) and also this Slack thread where @jonaslagoni is involved https://json-schema.slack.com/archives/CT8QRGTK5/p1637146426080000?thread_ts=1636976501.051400&cid=CT8QRGTK5

Meaning #649 is potentially gonna be closed as won't do if we decide indeed the new $ref behavior does not fit our needs.

More to come on other issues.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[smoya]

Not expected to be closed as completed 🤷. I created a PR for fixing that asyncapi/.github#174 cc @derberg