Proposal

[mistermoe]

Problem Statement

Many projects at TBD leverage json schemas. There's two primary use-cases:

1. protocols and open standards (e.g. tbdex, kcc, dwn etc.) use json schemas to define the structure and expected types of data models. These schemas are then used across many languages to validate said data.

2. JSON schemas can be used to provide intellisense and validation in IDEs like VSCode for configuration files (e.g. FTL toml files, helm chart templates etc.)

We've run into the following problems in our use of json schemas:

1. no single source of truth - schemas end up getting defined in one place and then copy pasta'd wherever they're needed. this makes it difficult to maintain said schemas
2. hosting - Some projects have taken a step towards single source of truth for all schemas by placing them in a single repo and using github pages to host them. This works great but is a bit cumbersome to maintain for every project.
3. versioning - there are many ways to version schemas, particularly when using git, but hosting all versions can be challenging.

Proposal

I'm proposing a single open source github repo that we can use to house and/or mirror all json schemas that we maintain across projects. We would then host this repo via github pages with a custom domain

Directory structure

The schemas repo would have the following directory structure:

❯ tree schemas
schemas <project root>
├── [namespace]
│   └── [version]
│       └── arbitrary.schema.json
└── hosted
    └── [namespace]
        └── [version]
            └── arbitrary.schema.json

Example

❯ tree schemas
schemas
├── dwn (@4c064165)
├── ftl
│   └── 0.281.3
│       └── project.schema.json
├── infra
│   └── 2024.03.10
│       └── rds.schema.json
├── kcc
└── tbdex (@3d098127)

Each namespace is a project, standard, or protocol that we maintain schemas for. Each namespace would have a directory for each version respective schemas.

Important

these namespace directories can also be git submodules for projects that want to maintain their schemas within their own repos. These repos MUST include a just schemas command that will output the expected directory structure into a .schemas directory in the root of the submodule.

Note

there is no mandated versioning scheme.

hosted

The hosted directory is the artifact that is created via a github action and hosted on github pages. This directory will also contain an index.html file that acts as a landing page and index for all namespaces.

Implementation Plan

• Add tbdex and kcc submodules #2
• Add Justfile to generate .schemas/ directory tbdex#352
• Update tbdex and kcc submodules #7
• Rename schemas file to match convention known-customer-credential#38
• Add GitHub Actions workflow to automate publishing of hosted schemas #3
• Add index.html for visual landing page #4
• Configure GitHub Pages with schemas.host custom domain #5
• Explore feasibility of sub-domain redirects #6
• Add README with a usage guide #8
• Add KBC schema known-customer-credential#40
• Add Justfile to copy schemas to .schemas/ known-customer-credential#41

[mistermoe]

something to take a look at: https://www.schemastore.org/json/

[csuwildcat]

DIF just started a Credential Schema initiative to host community schemas of all kinds, so we could take advantage of that.

[decentralgabe]

The trust framework will be responsible for hosting schemas within its domain, like the KCC schemas.

Similarly, when tbDEX is moved out from TBDs control the schemas will be hosted there--probably through whatever becomes of the tbdex.io site.

I have no problem starting with this proposal with the understanding that schemas will move when those things get figured out

[mistermoe]

DIF just started a Credential Schema initiative to host community schemas of all kinds, so we could take advantage of that.

@csuwildcat, nice! do you have a link that explains how it works

[frankhinek]

DIF just started a Credential Schema initiative to host community schemas of all kinds, so we could take advantage of that.

@csuwildcat would also be curious how soon this would be live if its a new initiative. In other words, is it available today/next 2 weeks or something we might consider a few months from now?

[frankhinek]

these namespace directories can also be git submodules for projects that want to maintain their schemas within their own repos. These repos MUST include a just schemas command that will output the expected directory structure into a .schemas directory in the root of the submodule.

@mistermoe presumably the intent behind requiring the just schemas command that outputs to a .schemas directory is to allow for consistent pathnames so that the GitHub action generating the hosted/ directory structure knows where to find the schema files. Is that correct and are there any other intentions behind this requirement?

[csuwildcat]

@frankhinek more info here: https://github.com/decentralized-identity/credential-schemas

[csuwildcat]

The DIF thing is basically a schema registry, which can catalog schemas that aren't necessarily under its domain.

[mistermoe]

these namespace directories can also be git submodules for projects that want to maintain their schemas within their own repos. These repos MUST include a just schemas command that will output the expected directory structure into a .schemas directory in the root of the submodule.

@mistermoe presumably the intent behind requiring the just schemas command that outputs to a .schemas directory is to allow for consistent pathnames so that the GitHub action generating the hosted/ directory structure knows where to find the schema files. Is that correct and are there any other intentions behind this requirement?

@frankhinek correct! no other intentions behind the requirements

[mistermoe]

DIF just started a Credential Schema initiative to host community schemas of all kinds, so we could take advantage of that.

@csuwildcat seems like its specific to just credentials. Have you heard that it's intent is to go beyond credential schemas?

[decentralgabe]

I don't see a problem with replicating our credential schemas to DIF, though I believe the associated projects should be the source of truth (whether its the protocol, the trust framework, etc.).

[mistermoe]

Yeah I'm not concerned about replicating. Moreso just trying to figure out whether @csuwildcat is suggesting we use DIF's offering as a wholesale alternative to the proposal here.