Bundling schema with urn ID does not replace urn $ref

[andlbrei]

Hi,

I'm trying to bundle a schema with urn identifiers.
I added the urn plugin for @hyperjump/browser based on the example.
The external schema (Address) is added to $defs in the resulting schema, but the $ref is not changed to refer to $defs.
What am I doing wrong?

Combining the two following schemas:

{
	"$schema": "https://json-schema.org/draft/2020-12/schema",
	"$id": "urn:test:something/schemas/CompanyCertificate",
	"type": "object",
	"title": "Company Certificate",
	"description": "It's the Company Certificate",
	"properties": {
		"companyInformation": {
			"type": "object",
			"properties": {
				"businessAddress": {
					"$ref": "urn:test:something/schemas/Address"
				}
			}
		}
	}
}

{
	"$schema": "https://json-schema.org/draft/2020-12/schema",
	"$id": "urn:test:something/schemas/Address",
	"type": "object",
	"required": ["addressLine1", "postalCode", "postalPlace"],
	"properties": {
		"addressLine1": {
			"type": "string"
		},
		"postalCode": {
			"type": "string"
		},
		"postalPlace": {
			"type": "string"
		}
	}
}

With this code:

addUriSchemePlugin("urn", {
  parse: (urn, baseUri) => {
    let { nid, nss, query, fragment } = parseUrn(urn);
    nid = nid.toLowerCase();

    if (!mappings[nid]?.[nss]) {
      throw Error(`Not Found -- ${urn}`);
    }

    let uri = mappings[nid][nss];
    uri += query ? `?${query}` : "";
    uri += fragment ? `#${fragment}` : "";

    return retrieve(uri, baseUri);
  },
});

registerSchema(schema);
registerSchema(Address);

const bundledSchema = await bundle(
  "urn:test:something/schemas/CompanyCertificate"
);

Results in this:

{
	"$schema": "https://json-schema.org/draft/2020-12/schema",
	"type": "object",
	"title": "Company Certificate",
	"description": "It's the Company Certificate",
	"properties": {
		"companyInformation": {
			"type": "object",
			"properties": {
				"businessAddress": { "$ref": "urn:test:something/schemas/Address" }
			}
		}
	},
	"$defs": {
		"Address": {
			"$id": "Address",
			"type": "object",
			"required": ["addressLine1", "postalCode", "postalPlace"],
			"properties": {
				"addressLine1": { "type": "string" },
				"postalCode": { "type": "string" },
				"postalPlace": { "type": "string" }
			}
		}
	}
}

Thanks!

[jdesrosiers]

The external schema (Address) is added to $defs in the resulting schema, but the $ref is not changed to refer to $defs.

The $ref isn't supposed to change. You can reference an embedded schema by its identifier, so no references need to change.

However, there are a couple of unusual things happening here. First, I think it should be emitting a $id for the root schema in this case. Second, URNs aren't really supposed to be relative, so making the $id of the embedded schema relative isn't really good practice in this case even though it technically works.

But, despite a couple ways that things are awkward when using URNs (I'll work on it), the resulting schema should work just fine. Are you getting errors when trying to use the bundled schema?

[andlbrei]

Thanks for the quick reply!

I would have thought that the $id would have stayed a URN, or that the $ref would change.
Nice to know what the correct behaviour should be.

What I am trying to do is get TypeScript types from my JSON Schema.
Ajv is used for validating, and it works fine, but it does not have type inference for referenced schemas, which is understandable.
Having schemas defined in separate files is desired, since I will end up with a decent amount of schemas.
These schemas will be used several ways, some dynamically loaded, some might live on a file system, so I figured using URN for $id would work fine.
So I already have my schemas defined, and I need to jump through some hoops to get types from them.

I tried using json-schema-to-typescript, which uses json-schema-ref-parser under the hood to resolve references, but it couldn't find the other schemas, which makes sense as it doesn't have any way of registering schemas, so it can't resolve them based on the URN.
My strategy was then to bundle the schemas using @hyperjump/json-schema/bundle, and use the bundled schema to generate the types.

Running the result schema from my earlier post into json-schema-to-typescript gives an error message:

  stack: 'ResolverError: Error opening file "urn:test:something/schemas/Address" \n' +
    "ENOENT: no such file or directory, open 'urn:test:something/schemas/Address'\n" +
    '    at Object.read (/Users/andersbreilid/workspace/dnb-web-codd-apps/node_modules/.pnpm/@[email protected]/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/resolvers/file.js:61:19)',
  code: 'ERESOLVER',
  name: 'ResolverError',
  message: 'Error opening file "urn:test:something/schemas/Address" \n' +
    "ENOENT: no such file or directory, open 'urn:test:something/schemas/Address'",
  source: 'urn:test:something/schemas/Address',
  path: null,
  toJSON: [Function: toJSON],
  ioErrorCode: 'ENOENT',
  footprint: 'null+urn:test:something/schemas/Address+ERESOLVER+Error opening file "urn:test:something/schemas/Address" \n' +
    "ENOENT: no such file or directory, open 'urn:test:something/schemas/Address'",
  toString: [Function: toString]

This error is the same as I would get using a non-bundled schema, naturally, as it cannot resolve the URN.
I tried modifying the $id to be the full URN, as shown below, but it could still not resolve the reference.

{
	"$schema": "https://json-schema.org/draft/2020-12/schema",
	"$id": "urn:test:something/schemas/CompanyCertificate",
	"type": "object",
	"title": "Company Certificate",
	"description": "It's the Company Certificate",
	"properties": {
		"companyInformation": {
			"type": "object",
			"properties": {
				"businessAddress": { "$ref": "urn:test:something/schemas/Address" }
			}
		}
	},
	"$defs": {
		"Address": {
			"$id": "urn:test:something/schemas/Address",
			"type": "object",
			"required": ["addressLine1", "postalCode", "postalPlace"],
			"properties": {
				"addressLine1": { "type": "string" },
				"postalCode": { "type": "string" },
				"postalPlace": { "type": "string" }
			}
		}
	}
}

They don't seem to be working towards implementing the JSON Schema spec as indicated in this thread, where they state that it is really a OpenAPI 3.0 tool.
(In general there seems to be many tools made for OpenAPI, and many of them lack support for OpenAPI 3.1 and the introduced JSON Schema compatibility.)

You might be correct in that the resulting JSON Schema should work, but I cannot find a tool that can generate types from it, at least not when using URNs.
It seems that I will need to use a different $id in order to get types from my schemas.

[jdesrosiers]

Yeah, json-schema-ref-parser doesn't understand embedded schemas, so this bundler isn't going to help because it's based on the concept of embedded schemas. However, there's a different approach that might work for you.

Instead of using URNs, don't use use $id at all and allow the file system path to be schema's identifier. This implementation supports that out-of-the-box and it should work with json-schema-ref-parser, which I believe understands file-based references. Ajv doesn't understand file-based references, but you can bundle and then use the result in ajv.

So, I think the following should work in json-schema-ref-parser.

/path/to/schemas/company-certificate.schema.json

{
	"$schema": "https://json-schema.org/draft/2020-12/schema",
	"type": "object",
	"title": "Company Certificate",
	"description": "It's the Company Certificate",
	"properties": {
		"companyInformation": {
			"type": "object",
			"properties": {
				"businessAddress": {
					"$ref": "./address/schema.json"
				}
			}
		}
	}
}

/path/to/schemas/address.schema.json

{
	"$schema": "https://json-schema.org/draft/2020-12/schema",
	"type": "object",
	"required": ["addressLine1", "postalCode", "postalPlace"],
	"properties": {
		"addressLine1": {
			"type": "string"
		},
		"postalCode": {
			"type": "string"
		},
		"postalPlace": {
			"type": "string"
		}
	}
}

And if you bundle, it should work in ajv.

// No registering schemas needed
const bundledSchema =  await bundle("./schemas/company-certificate.schema.json");

// Then load into ajv and validate

Or you could use this library to do the validation without needing to bundle.

// No registering schemas needed
const result = await validate("./schemas/company-certificate.schema.json", subject);

[andlbrei]

Thanks, this seems like a good enough solution for now.
I would like to be able to load schemas dynamically without caring about file system references, but I think I can get around it in any case.

Thanks for quick and detailed help!