Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/json schema article #6787

Closed
wants to merge 16 commits into from
Closed

Conversation

pubkey
Copy link
Owner

@pubkey pubkey commented Jan 28, 2025

DO NOT MERGE

docs-src/docs/article-json-schema.md Outdated Show resolved Hide resolved
docs-src/docs/article-json-schema.md Outdated Show resolved Hide resolved
docs-src/docs/article-json-schema.md Outdated Show resolved Hide resolved
docs-src/docs/article-json-schema.md Show resolved Hide resolved
docs-src/docs/article-json-schema.md Show resolved Hide resolved
docs-src/docs/article-json-schema.md Outdated Show resolved Hide resolved

Over time, RxDB has evolved its usage of JSON Schema, learning from real production experiences and feedback from the community. Here are some key takeaways:

- Avoid inlined `required` fields: Some validators let you write `"required": true` inside the property definition, but the official JSON Schema specification requires you to declare an array of required fields at the parent object level. Following the standard from the start avoids confusion and ensures broader validator compatibility.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is historical. JSON Schema Draft 3 used the boolean form of required. The issue here really touches on what I mentioned before: most schema authors avoid $schema, thus set no versions, and everybody gets confused about what version of JSON Schema they are actually using. Can you revise this paragraph, maybe rephrasing it as some validators being obsolete? (as required: true was indeed valid ages ago!)


- Avoid inlined `required` fields: Some validators let you write `"required": true` inside the property definition, but the official JSON Schema specification requires you to declare an array of required fields at the parent object level. Following the standard from the start avoids confusion and ensures broader validator compatibility.

- Keep Custom Fields at the Top Level: Originally, RxDB allowed custom definitions (`index`, `encrypted`, etc.) to appear deeply nested. This caused performance hits because the library had to traverse large schema objects to find them. By placing these fields at the top level, RxDB can parse and apply them much faster, improving startup times.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I now wonder if these custom fields even need to be part of the schema. Maybe you have a JSON object with these properties and a schema property? Then you don't have to deal with the complexity of creating a custom JSON Schema vocabulary, etc

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will think about this. Maybe RxDB should move them to somewhere else.


- Keep Custom Fields at the Top Level: Originally, RxDB allowed custom definitions (`index`, `encrypted`, etc.) to appear deeply nested. This caused performance hits because the library had to traverse large schema objects to find them. By placing these fields at the top level, RxDB can parse and apply them much faster, improving startup times.

- Error messages are not standardized: Each validator produces a different structure for error messages. If your app logic inspects these errors, you risk partial or complete rewrites if you ever switch validators. Decide early on which validator meets your needs and plan on sticking with it long-term.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a huge one we are constantly battling against. Can you mention the Standard Output Formats I touched on before? If you can express your desire for validators to comply, that would be wonderful!

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I add this.

docs-src/docs/article-json-schema.md Outdated Show resolved Hide resolved
@jviotti
Copy link

jviotti commented Jan 29, 2025

Awesome stuff! I left a few comments just to polish things a bit. I also left some suggestions for improving RxDB's JSON Schema integration. Of course, those don't have to be solved for the case study, but would love to separately collaborate on them :)

Once we feel the draft is ready, we can send a PR to the official JSON Schema website to get reviews from the rest of the TSC and get it published.

Thanks a lot for putting this together! It already looks very interesting

@pubkey
Copy link
Owner Author

pubkey commented Jan 29, 2025

@jviotti Thank you for the feedback. I updated all parts, please check.


No single validator is perfect; each has its trade-offs in performance, build size, and correctness edge cases. Some libraries rely on `eval()` or `new Function`, which can break in strict Content-Security-Policy (CSP) environments. Others have issues with large integers. RxDB encourages you to pick a validator that matches your app's needs and environment constraints.

Notice that RxDB itself does not define a specific JSON Schema draft. Instead it fully depends on the validation plugin is use, which parts of the JSON schema spec are accepted.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph doesn't read very well. Maybe:

Note that RxDB itself does not enforce the use of a specific JSON Schema dialect. Instead, the range of supported dialects is determined by the JSON Schema validation plugin in use.

Also BTW, we tend to say "dialect" instead of "draft". The "draft" terminology is just old IETF annoyance

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replaced "draft" with dialect everywhere.


No single validator is perfect; each has its trade-offs in performance, build size, and correctness edge cases. Some libraries rely on `eval()` or `new Function`, which can break in strict Content-Security-Policy (CSP) environments. Others have issues with large integers. RxDB encourages you to pick a validator that matches your app's needs and environment constraints.

Notice that RxDB itself does not define a specific JSON Schema draft. Instead it fully depends on the validation plugin is use, which parts of the JSON schema spec are accepted.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph doesn't read very well. Maybe:

Note that RxDB itself does not enforce the use of a specific JSON Schema dialect. Instead, the range of supported dialects is determined by the JSON Schema validation plugin in use.

Also BTW, we tend to say "dialect" instead of "draft". The "draft" terminology is just old IETF annoyance


No single validator is perfect; each has its trade-offs in performance, build size, and correctness edge cases. Some libraries rely on `eval()` or `new Function`, which can break in strict Content-Security-Policy (CSP) environments. Others have issues with large integers. RxDB encourages you to pick a validator that matches your app's needs and environment constraints.

Notice that RxDB itself does not define a specific JSON Schema draft. Instead it fully depends on the validation plugin is use, which parts of the JSON schema spec are accepted.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph doesn't read very well. Maybe:

Note that RxDB itself does not enforce the use of a specific JSON Schema dialect. Instead, the range of supported dialects is determined by the JSON Schema validation plugin in use.

Also BTW, we tend to say "dialect" instead of "draft". The "draft" terminology is just old IETF annoyance


No single validator is perfect; each has its trade-offs in performance, build size, and correctness edge cases. Some libraries rely on `eval()` or `new Function`, which can break in strict Content-Security-Policy (CSP) environments. Others have issues with large integers. RxDB encourages you to pick a validator that matches your app's needs and environment constraints.

Notice that RxDB itself does not define a specific JSON Schema draft. Instead it fully depends on the validation plugin is use, which parts of the JSON schema spec are accepted.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph doesn't read very well. Maybe:

Note that RxDB itself does not enforce the use of a specific JSON Schema dialect. Instead, the range of supported dialects is determined by the JSON Schema validation plugin in use.

Also BTW, we tend to say "dialect" instead of "draft". The "draft" terminology is just old IETF annoyance


Over time, RxDB has evolved its usage of JSON Schema, learning from real production experiences and feedback from the community. Here are some key takeaways:

- Avoid inlined `required` fields: Some validators let you write `"required": true` inside the property definition because that was part of JSON Schema Draft 3. But the newer JSON Schema drafts require you to declare an array of required fields at the parent object level. Following the standard from the start avoids confusion and ensures broader validator compatibility.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it reads better now, but we probably should say to "avoid inlined required fields". Previously in the article, you say the JSON Schema dialect in use depends on the user plugin, so in theory some of them might use a plugin that is intentionally Draft 3, and thus this advice wouldn't make sense.

Maybe you can rephrase the advise as: make sure you know which JSON Schema dialect is in use to avoid confusion, citing inlined required and Draft 3 as an example of such confusion?

@jviotti
Copy link

jviotti commented Jan 30, 2025

I left a few more comments, but I think it's already pretty good. Can you send a PR to https://github.com/json-schema-org/website including it, and we get a final review from the entire TSC?

pubkey added a commit that referenced this pull request Jan 31, 2025
@pubkey
Copy link
Owner Author

pubkey commented Jan 31, 2025

@jviotti Thank you. I changed the parts and added a PR to the website: json-schema-org/website#1385

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants