Add OpenAPI json file

[Gr3q]

This PR attempts to add an openapi.json to the project for client library/documentation generation. This is a big one.

The file should be usable but I will try generating a Python client library soon to see how well it works, but work on the PR can be started even now. Ideally all extension should be added the schema. So far I added extension endpoints, parameters, responses, http post endpoints.

It's Github, there is a live URL to the json file, so technically it can be used for stuff straight away...

Everything below will need to be resolved one way or another, they need decisions by the maintainers of the API specification..

Required work to be done before merge

• Split out all endpoints, responses and component schemas out to files matching the folder structure using relative path refs. All files should resolve properly in editors for ease of editing.
• Add build command that resolves all relative refs (and only those) properly handling nested refs
  • If top level ref: dumps contents into final openapi.json
  • If nested ref: convert into #/... format (final place in the openapi.json should be inferred from the file path alone, e.g. /content/en/docs/Responses/ArtistID3.json goes to #/components/schemas/ArtistID3)
• Add step to validate final openapi.json during build and for PRs
• Integrate openapi.json schema building into hugo somehow
• OpenAPI online (Swagger/Redoc) documentation generated along with the API spec

Work checklist(s)

Decide to Approve/Change/Fix every difference between openapi.json and OpenSubsonic API spec

• Some typo fixes I failed to document :( I will need to find them again and add them to the section below
• Added minimum: 0 to integer types where it's implied (count, offset, unix timestamp, position)
• forces json format on every endpoint. xml response formats should not be a big hassle but client library generation will probably become problematic.
• All extensions are added to the schema and tagged as "Extension", and have a 404 return type as well that described as "Not Implemented"
• Excluded examples, opted to use externalDocs only because it made the Swagger and Redoc generated documents harder to read. I might add them if they make sense for generated client code.
• Query Parameters only existing via Extensions are always added and marked in their description field
• HTTP form POST extension support might be stricter than what's allowed (global params - auth and format params - only allowed as query params, endpoint specific params are the only ones allowed in request body)

Decide to Schedule/Postpone/Fix/Resolve all issues/inconsistencies found in spec

They might need to be resolved before/along with this PR or they can be resolved at a later time. It would be nice if all of them could be answered.

• ItemDate description and spec doesn't match (in required fields)
• Disc number must be a number but obviously it can be other things
• getAlbumInfo2 return type incorrect (based on AlbumList)?
• downloadPodcastEpisode return type incorrect?
• deleteBookmark summary and description is wrong
• download error payload schema missing completely
• getArtists page first artists link is broken
• getAvatar username parameter is "required" and has a "default" as well. it should be one or the other
• getCaptions return payload not clear
• getCoverArt size param type not specified, I assume int as pixels
• Indexes ignoredArticles pattern is unclear, it was comma-separated list as string before
• internetRadioStation homePageUrl has wrong details
• getLyrics artist and title query parameters are really optional?
• is getLyrics returns only one element?
• Extension not supported error is unclear, assumed servers will 404 on the request.
• PodcastEpisode description "PodcastEntry extends Child". Wrong name used?
• NowPlayingEntry minutesAgo format is unclear
• PlayQueue description incorrect
• PodcastChannel errorMessage existence could depend on status in documentation?
• GetSimilarSongs and GetSimilarSongs2 response type is the same in the spec (not the names)?
• User example uses booleans and integers as strings, but they are typed as booleans and integers.
• VideoInfo is not documented
• Videos is not documented
• hls endpoint summary matches download
• Typo in search album parameters's comment
• search any parameter type unclear
• SearchResult is not documented
• SearchResult2 all field details are incorrect
• stream timeOffset param's description typo, "Transcode Offet"
• tokenInfo data is too deeply nested in docs?
• tokenInfo says "A subsonic-response element with a nested tokenInfo on success, or error 44 on invalid token.". Does that mean it cannot return any other error code on error?
• unstar summary is incorrect
• updateuser is missing playlistRole parameter?
• createuser is missing maxBitRate parameter?

Schedule/Require/Postpone/Request future improvements for openapi.json

• Could use inheritance for ArtistInfo and ArtistInfo2
• getPodcasts payload dependent on query param includeEpisodes, but it doesn't indicate that structurally.
• jukeboxControl payload dependent on query param action: get, but it doesn't indicate that structurally.
• getAlbumList and getAlbumList2 POST params can be a tagged union
• jukeboxcontrol POST params can be a tagged union
• Add version info for each endpoint and add 404 responses where the endpoint didn't exist since v1
• add (back) examples

Review generated OpenAPI documentation to verify they are OK

• SwaggerUI
• ReDoc

Verify this OpenAPI schema against servers

Using at least one of the tools from here the openapi.json should be tested against all endpoints request/response payloads on the servers below. Any mismatches/errors should be fixed if openapi.json doesn't follow the spec. Ideally will be no behaviors implemented in servers that are different in the official API spec.

Way out of scope.

Client libraries generated from this openapi.json are generated OK

Python

• synchronous library
• async library
• library using pydantic for response payload validation - no such library found

Proposed client generation libraries for testing

• https://github.com/openapi-generators/openapi-python-client
• https://openapi-generator.tech/docs/generators/python/

Typescript

• Working async library
• Working library using zod for response payload validation

Proposed client generation libraries for testing

• https://www.npmjs.com/package/openapi-typescript
• https://www.npmjs.com/package/@openapitools/openapi-generator-cli (not really good but official)
• https://www.npmjs.com/package/openapi-zod-client

Kotlin

• Working library
• Use library against Navidrome and test every endpoint
• Use library against any server requested by maintainers and test every endpoint

Proposed client generation libraries for testing

• https://openapi-generator.tech/docs/generators/kotlin/

Any other language required by maintainers

• Dart?
• C#?

Major Future Work

• Documentation could be generated from the openapi.json (in its current form)

[netlify]

✅ Deploy Preview for opensubsonic ready!

Name	Link
🔨 Latest commit	f4d0c81
🔍 Latest deploy log	https://app.netlify.com/projects/opensubsonic/deploys/682ce4895f11760008d3e9ee
😎 Deploy Preview	https://deploy-preview-137--opensubsonic.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Tolriq]

As explained before IMO

Break up openapi.json to smaller chunks - and add build step to merge and rebuild it into one file that can be served.

Should be done first, PR merging, rebase and everything will be a pain with such a single large file.

There's tons of tools to rebuild the large file from the split version.

[Gr3q]

Should be done first, PR merging, rebase and everything will be a pain with such a single large file.

For the purposes of this PR it will probably be done last, breaking it up now will slow down making sweeping changes (if requested) and testing it's validity.

[Tolriq]

Your future work was not clear if it was tied to the PR or for future PR. If the split is made before it's ready to merge then no problem.

[Gr3q]

Your future work was not clear if it was tied to the PR or for future PR. If the split is made before it's ready to merge then no problem.

It's a PR, it's meant to be reviewed and changed to fit the needs of the project.

Although yes, I hoped I won't need to write GitHub actions and a build step for the file for the first version as it probably won't be that simple if we want client (and server) generation work nicely (which is my priority).

[Tolriq]

(which is my priority).

The priority is actually that anything that is merged is stable and maintainable, this is a public documentation of a public API used by many servers and clients, breaking changes must be avoided at all costs.

If we go single file, but can't split because it would trigger some small incompatible changes we are doomed.

One solution could be to publish the OpenAPI spec as alpha / unsupported until we are ready to commit to it, but the priority is stability not rushing.

The generation does not need to be a github action, the project is already provided with docker and it can be a simple command in it to generate the json, this comes with the added benefits of content validation at that step to avoid PR with errors in the files, this also avoids all formatting issues, greatly simplifies future reviews, ...

[Gr3q]

You missed the part where I changed the PR description to require your requested changes.

I can reiterate that I wrote before.

It's a PR, it's meant to be reviewed and changed to fit the needs of the project.

My motivation and priorities are not something that will change (for this API to be easier to use and adopt) but it also won't affect this PR.

The generation does not need to be a github action, the project is already provided with docker and it can be a simple command in it to generate the json, this comes with the added benefits of content validation at that step to avoid PR with errors in the files, this also avoids all formatting issues, greatly simplifies future reviews, ...

I simply meant some form of pipeline step is needed on every PR to preserve the structural validity of the final generated JSON.

[Tolriq]

I simply meant some form of pipeline step is needed on every PR to preserve the structural validity of the final generated JSON.

In all cases we need a way to validate the file(s) if there's one or many that does not change the need, else how to you review the PRs? Humans can't spot a typo or a missing comma in a large commit.

[Tolriq]

@Gr3q Do you intend to still work on this ?

[Gr3q]

Yes, when I have time.

What about the huge checklists below the "Required work to be done before merge" section?

I can take the lack of comments on them that they are not important for you guys for this PR?

[Tolriq]

Most don't care about the doc part details so it's hard to get engagement until it's finished.

For my case, this work was planned to be done by me to unlock the new json endpoints so it's very important, and your list of required correspond to what I requested here and would have done so not much to comment.

Edit: Sorry for close missclick

[Gr3q]

@Tolriq which folder structure and target folder do you prefer for the broken out schemas?

Contextual requirements/options

• Target Folder options:
  • content/en/docs
  • openapi
  • anywhere you want really
• The base openapi.json should be above the target folder so path traversals are not needed for it's references (No "$ref": "../{schemapath}"
• All broken out schema filenames (responses, endpoints etc) should match the original documentation filename in content/en/docs

Options

Options below are using Subsonic Response, content/en/docs as target folder and openapi.json placed into the root folder as an example.

Option one: flat list placed directly into the target folder/schema type

openapi.json
content/en/docs/Responses/
   ...
   Subsonic-baseResponse.json
   Subsonic-failureResponse.json
   Subsonic-response.json
   Subsonic-response.md
   Subsonic-successResponse.json
   ...

Option 2: Supplementary schemas go into subfolder of the same name

openapi.json
content/en/docs/Responses/
   ...
   Subsonic-response/
      Subsonic-baseResponse.json
      Subsonic-failureResponse.json
      Subsonic-successResponse.json
   Subsonic-response.json
   Subsonic-response.md
   ...

Option 3: All json shemas go into a subfolder of the same name

openapi.json
content/en/docs/Responses/
   ...
   Subsonic-response/
      Subsonic-baseResponse.json
      Subsonic-failureResponse.json
      Subsonic-successResponse.json
      Subsonic-response.json
   Subsonic-response.md
   ...

[Tolriq]

IMO it should be under an openapi folder to not mix with the .md files. Because for now it's beta stuff and in the far future as you said we could generate the md files from the openapi files and so it's better not to mix them in the hierarchy.

For the folders under openapi I have no strong opinion except that it's better if we have some kind of minimal organization.

I don't know the best practices here as I still have not had found time to fully investigate this stuff.

[Tolriq]

Thanks we'll still need to find a way to update docsy to a more recent version too (Dark mode :) ) and fix the not visible versions in white over white.

[Gr3q]

Thanks we'll still need to find a way to update docsy to a more recent version too (Dark mode :) ) and fix the not visible versions in white over white.

You will still need the fix for swagger but here you go #147

[Tolriq]

You will still need the fix for swagger

Yes but it would be a small update.
Anyway thanks again a lot for the amazing work you've done here and the other PR to migrate all changes.

I'll now be able to work on the new APIs without having to deal with all those and it's a massive headache removed.

[Tolriq]

Ok so @Gr3q IMO it looks good and works well enough to have this as a basis for the future.

I think that to merge in current status we just need to open a discussion and move the TODO list there.

And add an red important box in the https://deploy-preview-137--opensubsonic.netlify.app/docs/openapi/ page saying that this is still a WIP as there's some inconsistencies and users should still check the actual docs and servers results until this is finished and a link to the discussion TODO list.

WDYT ?

[Gr3q]

@Tolriq will do, @deluan also said to do that, I simply forgot.

[Tolriq]

Should we extract the todo from this PR to GitHub discussions to simplify future management and not be editing a closed PR ?

[Tolriq]

Seems swagger / redoc does not work correctly in dark mode :(

[Gr3q]

Seems swagger / redoc does not work correctly in dark mode :(

• Added a white background for redoc
• Added dark theme support for swagger

Adding a full swagger theme is overkill in my opinion, but I couldn't get hugo use dart-sass no matter what I tried (to use @use directives in scss that I could use to do scoped imports).

The alternative is to remove the dark theme for swagger and add minimal styling (white bg) it looks the same in dark mode too. I'm not sure why no-one is complaining in the docsy github repo about this.

[Tolriq]

Yes no need to make something complicated, just that it's legible for quick checks when needed.

People can and will use directly the json in their preferred tool for extended usage.

I think it's now in a working shape to be merged since it's labelled as WIP, this will make other react more on the small details / differences.

Do you see anything missing ?

[Gr3q]

I think it's ready.

I would love to resolve the TODOs, otherwise I consider the schema ready for this PR. If I forgot to write something down in the dev docs for it I wouldn't know...so for now that is also ready,

[Tolriq]

Don't worry when building the first version of this docs, I made a couple mistakes as it was a huge dumb work, and at some point the brain no more print :)

We'll fix with reports.

[Tolriq]

@opensubsonic/servers @opensubsonic/clients can we have some love here ? This is the basis for a better documentation and precise API definition for all the future new endpoints.

[paulijar]

I have no prior knowledge of OpenAPI or Swagger or Redoc, but overall, looks very fancy. Apparently the Redoc still isn't quite readable when the browser uses the dark mode?

Also, the Redoc UI navigation pane on the left doesn't seem to work at all?

[lachlan-00]

a bit beyond me but swagger is def a good focus point

[Tolriq]

looks very fancy

This is a side effect, the main thing is very precise API definition, allowing to verify that endpoints and clients are compliant without trying to check a webpage with markdown tables and find what mistake we made :)

Will merge as it's WIP but will ensure that there's no need to rebase non stop and we can fix the small details over time.

This also means people needs to update the OpenAPI part for new PRs.

If we can't fix redoc we can remove it too, the important part is the actual OpenAPI json definition, people can use other tools with the file.

[Gr3q]

I have no prior knowledge of OpenAPI or Swagger or Redoc, but overall, looks very fancy. Apparently the Redoc still isn't quite readable when the browser uses the dark mode?

I will fix that sometime, I didn't notice it before.

Also, the Redoc UI navigation pane on the left doesn't seem to work at all?

Docsy has some custom handling for the sidebar, it probably doesn't work for some reason.