Improve JSON Schema output from the parser #562

KonstantinSviridov · 2016-11-30T17:53:02Z

We've got a lot of very good feedback about the current JSON object that comes back from the parser. Most predominantly, the output should be more consistent.

This issue describes the different changes we'd like to make. Please feel free to help and discuss in this matter.

Improvements

Schema Field	Current Type	Change	RAML version
`mediaType`	Its either a `string` or an `array`	Only `array`	1.0
`types`	An `array` of single key objects	`map`	1.0
`traits`	An `array` of single key objects	`map`	1.0, 0.8
`resourceTypes`	An `array` of single key objects	`map`	1.0, 0.8
`annotationTypes`	An `array` of single key objects	`map`	1.0
`securitySchemes`	An `array` of single key objects	`map`	1.0, 0.8
`schemas`	Is serialized under the `schemas` key	serialize under the `types` key	1.0
`schema`	Is serialized under the `schema` key	serialize under the `type` key	1.0
`example`	Is serialized under the `example` key	serialize as one element array under the `examples` key	1.0
`value` (ExampleSpec)	string representation of example	actual example value: `string`, `object`, `array`, 'number', `boolean` or `null`	1.0

New

Field	Type	Description	RAML version
`completeRelativeUri`	`string`	Resource URI relative to Api root. Adding it to any resource as a `string` value.	1.0, 0.8
`parentUrl`	`string`	For a nested resource it's the complete relative URI of its parent. For a top level resource it's an empty string. Adding it to any resource as a `string` value.	1.0, 0.8
`allUriParameters`	`array`	A union of URI parameters sets of the resource itself with URI parameters sets of all the resources which contain the resource. Adding it to any resource as an optional `array`.	1.0, 0.8
`allUriParameters`	`array`	`allUriParameters` value of the owning resource. Adding it to any method as an optional `array`.	1.0, 0.8
`mediaType` (for Type)	`string`	States the way of type definition: `application/json` or `application/xml` for external types and `application/raml+yaml` for the rest	1.0

Remove

structuredExample
structuredValue (ExampleSpec)

The text was updated successfully, but these errors were encountered:

sichvoge · 2016-12-01T12:55:28Z

QQ: Should we also handle and transform all type expressions and give a simple to read and consistent output? I know that parsing things like [Cow, Cat | Dog] might be easy, but the logic to parse them to get a more appropriate format seems to be replicated on clients.

sichvoge · 2016-12-01T12:59:04Z

@KonstantinSviridov can you guys investigate if there are other nodes that can be expressed differently like:

type: Dog[]

type: array
items: Dog

mediaType: application/json

mediaType: [ application / json ]

I am sure there are others.

sichvoge · 2016-12-01T13:13:18Z

Others are

types:
  test:
    type: object
    properties:
      name: string
    example:
      value: !include examples/actionsRead.json

types:
  test:
    type: object
    properties:
      name: string
    example: !include examples/actionsRead.json

sichvoge · 2016-12-01T13:22:37Z

I'd also suggest to introduce a mediaType tag for types / schemas that clearly identifies the value type for a type value. For example:

{
  "type": {
    "mediaType": "application/raml+yaml",
    "content": {
      "type": "array",
      "items": "Dog"
    }
  }
}

Given the mediaType, I know exactly how to parse the content. Might be helpful and its just an idea. Would love to see what others think.

KonstantinSviridov · 2016-12-01T18:37:18Z

@sichvoge

Why would we put actual type definition into the "content" field. May be lets just introduce an additional "mediaType" field:

    "types": {
      "Account": {
        "name": "Account",
        "displayName": "Account",
        "mediaType": "application/xml",
        "type": [
          "<xs:schema attributeFormDefault=\"unqualified\" elementFormDefault=\"qualified\" xmlns:xs=\"http://www.w3.org/2001/XMLSchema\">\n    <xs:element name=\"TwilioResponse\">\n        <xs:complexType>\n            <xs:sequence>\n                <xs:element name=\"Account\">\n                    <xs:complexType>\n                        <xs:sequence>\n                            <xs:element type=\"xs:string\" name=\"Sid\"/>\n                            <xs:element type=\"xs:string\" name=\"FriendlyName\"/>\n                            <xs:element type=\"xs:string\" name=\"Type\"/>\n                            <xs:element type=\"xs:string\" name=\"Status\"/>\n                            <xs:element type=\"xs:string\" name=\"DateCreated\"/>\n                            <xs:element type=\"xs:string\" name=\"DateUpdated\"/>\n                            <xs:element type=\"xs:string\" name=\"AuthToken\"/>\n                            <xs:element type=\"xs:string\" name=\"Uri\"/>\n                            <xs:element name=\"SubresourceUris\">\n                                <xs:complexType>\n                                    <xs:sequence>\n                                        <xs:element type=\"xs:string\" name=\"AvailablePhoneNumbers\"/>\n                                        <xs:element type=\"xs:string\" name=\"Calls\"/>\n                                        <xs:element type=\"xs:string\" name=\"Conferences\"/>\n                                        <xs:element type=\"xs:string\" name=\"IncomingPhoneNumbers\"/>\n                                        <xs:element type=\"xs:string\" name=\"Notifications\"/>\n                                        <xs:element type=\"xs:string\" name=\"OutgoingCallerIds\"/>\n                                        <xs:element type=\"xs:string\" name=\"Recordings\"/>\n                                        <xs:element type=\"xs:string\" name=\"Sandbox\"/>\n                                        <xs:element type=\"xs:string\" name=\"SMSMessages\"/>\n                                        <xs:element type=\"xs:string\" name=\"Transcriptions\"/>\n                                    </xs:sequence>\n                                </xs:complexType>\n                            </xs:element>\n                        </xs:sequence>\n                    </xs:complexType>\n                </xs:element>\n            </xs:sequence>\n        </xs:complexType>\n    </xs:element>\n</xs:schema>"
        ]
      },
      "SomeType": {
        "name": "SomeType",
        "displayName": "SomeType",
		"mediaType": "application/raml+yaml",
        "type": [
          "object"
        ],
        "properties": {
          "name": {
            "name": "name",
            "displayName": "name",
            "type": [
              "string"
            ]
          }
        },
        "examples": [
          {
            "value": "{\n  \"name\": \"somestring\"\n}",
            "strict": true,
            "name": null,
            "structuredValue": {
              "name": "somestring"
            }
          }
        ]
      }
    },

sichvoge · 2016-12-01T18:41:12Z

Looks ok for me too :) BTW, looking at your example:

"examples": [
  {
    "value": "{\n  \"name\": \"somestring\"\n}",
    "strict": true,
    "name": null,
    "structuredValue": {
      "name": "somestring"
    }
  }
]

Whats the difference between value and structuredValue?

KonstantinSviridov · 2016-12-01T18:42:56Z

As for exmaple, everything is going to be fine, because we're going to serialize an erray of ExampleSpec objects which look exactly the same disregarding whether one uses

    example:
      value: !include examples/actionsRead.json

or

    example: !include examples/actionsRead.json

KonstantinSviridov · 2016-12-01T18:53:24Z

I've realized that we can not use maps for allUriParameters as we can come across several resources utilizing parameters with same names:

/res1{id}:
  /res2{id}:

Thus, we have to use arrays here.

KonstantinSviridov · 2016-12-01T18:58:04Z

@sichvoge

Whats the difference between value and structuredValue?

value is a string representation
structuredValue is what has actually been specified: number, string, boolean or object

KonstantinSviridov · 2016-12-01T19:21:42Z

As for TypeDeclaration.type property, I propose the following schema for it:

{
  "type" : "array",
  "items": {
     "oneOf": [
        { "type": "string", "description": "type name or schema content (for external types)" },
        { "ref": "#path/to/TypeDeclaration/definition" },
        { "ref": "#path/to/ObjectTypeDeclaration/definition" },
        { "ref": "#path/to/UnionTypeDeclaration/definition" },
        etc
     ]
  }
}

Thus, in case of an inline type

types:
  T1:
    type:
      properties:
        p1:

we'll have

"types": [
      {
        "T1": {
          "type": [
            {
              "type": [
                 "object"
              ],
              "properties": {
                "p1": {
                  "type": [
                    "string"
                  ]
                }
			  }
            }
          ]
        }
      }
    ]

At the same time, type: [Cow, Cat | Dog] could be serialized to

"type": [
  "Cow",
  {
    "type": "union",
    "components": [ "Cat", "Dog" ]
  }
]

, where UnionTypeDeclaration.components has the same type as TypeDeclaration.type

sichvoge · 2016-12-02T10:00:09Z

With regards to the string value: Do we really need that? Isn't it very easy for the client to stringify the structured output?

sichvoge · 2016-12-02T10:03:16Z

With regards to union. We actually already have a certain format for that raml-org/raml-spec#541. We just haven't had much time to merge it yet, but the output could already follow that model. So Cat | Dog can be represented also as:

type: union
anyOf: [ Cat, Dog ]

KonstantinSviridov · 2016-12-05T12:37:25Z

Ok, it seems that we do not need a separate field for string value. Lets remove value and then rename structuredValue to value.

As for union types format, lets use anyOf then. For my example

type: [Cow, Cat | Dog]

we will have

"type": [
  "Cow",
  {
    "type": "union",
    "anyOf": [ "Cat", "Dog" ]
  }
]

KonstantinSviridov · 2016-12-05T12:38:56Z

Note that UnionTypeDeclaration.anyOf is likely to have just the same type as TypeDeclaration.type

KonstantinSviridov · 2016-12-06T20:40:15Z

@sichvoge
As for those type objects, which are obtained by expressions parsing, I propose text/plain as their mediaType. For exmaple:

type: [Cow, Cat | Dog]

will be

{
  "type": [
    "Cow",
    {
      "type": "union",
      "anyOf": [ "Cat", "Dog" ],
      "mediaType" : "text/plain"
    }
  ],
  "mediaType" : "application/raml+yaml"
  
}

KonstantinSviridov · 2016-12-06T20:43:03Z

The listed changes are implemented in #570

sichvoge · 2016-12-08T12:47:53Z

Thanks @KonstantinSviridov. Can you please add what we discussed about structuredValue and value? Lets forget about the type expression resolving for now.

KonstantinSviridov · 2016-12-08T13:26:59Z

@sichvoge Thanks for reminding about value and structuredValue.

Actually, I've already implemented type expressions resolving. Should I remove it?

KonstantinSviridov · 2016-12-08T13:41:43Z

For example,
https://github.com/raml-org/raml-js-parser-2/blob/tck-schema-changes/src/raml1/test/data/TCK/raml-1.0/Types/Type%20Expressions/test008/api.raml#L19

becomes

https://github.com/raml-org/raml-js-parser-2/blob/tck-schema-changes/src/raml1/test/data/TCK/raml-1.0/Types/Type%20Expressions/test008/api-tck.json#L136-L148

sichvoge · 2016-12-08T15:02:54Z

Actually, I've already implemented type expressions resolving. Should I remove it?

Yes please remove it again. Only the items that has been put into the OP (first comment) has been agreed on.

ddenisenko · 2016-12-09T01:40:11Z

I guess this one can be closed. Please reopen if there are still matters to discuss.

KonstantinSviridov assigned sichvoge and KonstantinSviridov Nov 30, 2016

KonstantinSviridov mentioned this issue Nov 30, 2016

Spike on the raml2obj schema #491

Closed

ddenisenko added Critical enhancement labels Dec 1, 2016

ddenisenko added this to the Sprint 6 core milestone Dec 1, 2016

sichvoge changed the title ~~TCK JSON Schema changes~~ Improve JSON Schema output from the parser Dec 1, 2016

KonstantinSviridov mentioned this issue Dec 6, 2016

Tck schema changes #570

Closed

sichvoge mentioned this issue Dec 8, 2016

Annotations Structure raml-org/raml-tck#25

Closed

ddenisenko closed this as completed Dec 9, 2016

ddenisenko modified the milestones: Sprint 6 core, February 2016 Feb 10, 2017

ddenisenko mentioned this issue Mar 8, 2017

Reduce parser complexity and improving usability #504

Closed

22 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve JSON Schema output from the parser #562

Improve JSON Schema output from the parser #562

KonstantinSviridov commented Nov 30, 2016 •

edited

Loading

sichvoge commented Dec 1, 2016

sichvoge commented Dec 1, 2016 •

edited

Loading

sichvoge commented Dec 1, 2016

sichvoge commented Dec 1, 2016 •

edited

Loading

KonstantinSviridov commented Dec 1, 2016 •

edited

Loading

sichvoge commented Dec 1, 2016 •

edited

Loading

KonstantinSviridov commented Dec 1, 2016 •

edited

Loading

KonstantinSviridov commented Dec 1, 2016

KonstantinSviridov commented Dec 1, 2016 •

edited

Loading

KonstantinSviridov commented Dec 1, 2016 •

edited

Loading

sichvoge commented Dec 2, 2016

sichvoge commented Dec 2, 2016

KonstantinSviridov commented Dec 5, 2016

KonstantinSviridov commented Dec 5, 2016

KonstantinSviridov commented Dec 6, 2016

KonstantinSviridov commented Dec 6, 2016

sichvoge commented Dec 8, 2016

KonstantinSviridov commented Dec 8, 2016 •

edited

Loading

KonstantinSviridov commented Dec 8, 2016

sichvoge commented Dec 8, 2016

ddenisenko commented Dec 9, 2016

Improve JSON Schema output from the parser #562

Improve JSON Schema output from the parser #562

Comments

KonstantinSviridov commented Nov 30, 2016 • edited Loading

sichvoge commented Dec 1, 2016

sichvoge commented Dec 1, 2016 • edited Loading

sichvoge commented Dec 1, 2016

sichvoge commented Dec 1, 2016 • edited Loading

KonstantinSviridov commented Dec 1, 2016 • edited Loading

sichvoge commented Dec 1, 2016 • edited Loading

KonstantinSviridov commented Dec 1, 2016 • edited Loading

KonstantinSviridov commented Dec 1, 2016

KonstantinSviridov commented Dec 1, 2016 • edited Loading

KonstantinSviridov commented Dec 1, 2016 • edited Loading

sichvoge commented Dec 2, 2016

sichvoge commented Dec 2, 2016

KonstantinSviridov commented Dec 5, 2016

KonstantinSviridov commented Dec 5, 2016

KonstantinSviridov commented Dec 6, 2016

KonstantinSviridov commented Dec 6, 2016

sichvoge commented Dec 8, 2016

KonstantinSviridov commented Dec 8, 2016 • edited Loading

KonstantinSviridov commented Dec 8, 2016

sichvoge commented Dec 8, 2016

ddenisenko commented Dec 9, 2016

KonstantinSviridov commented Nov 30, 2016 •

edited

Loading

sichvoge commented Dec 1, 2016 •

edited

Loading

sichvoge commented Dec 1, 2016 •

edited

Loading

KonstantinSviridov commented Dec 1, 2016 •

edited

Loading

sichvoge commented Dec 1, 2016 •

edited

Loading

KonstantinSviridov commented Dec 1, 2016 •

edited

Loading

KonstantinSviridov commented Dec 1, 2016 •

edited

Loading

KonstantinSviridov commented Dec 1, 2016 •

edited

Loading

KonstantinSviridov commented Dec 8, 2016 •

edited

Loading