Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Describing operations and templates with SHACL #17

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Describing operations and templates with SHACL #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 181 additions & 0 deletions api-documentation/shacl.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# Using SHACL to describe data structures

## Story

As an API publisher

I want to use SHACL to describe data structures

So that consumers can use existing tooling to consume the API

## Details

SHACL, which stands for Shapes Constraint Language is an RDF vocabulary, defined in a [W3C recommendation][shacl], which is used to describe graph shapes.

It is very expressive, more rich than `hydra:Class` et.al., and out of the box provides multiple features to provide an accurate description of request payloads.

[Many uses for SHACL have been proposed][ucr] such as, but not limited to, validation and building user interfaces dynamically.

This page shows how Shapes can be used to replace or extend Hydra Core vocabulary terms within the API Documentation and resource representations.

[shacl]: https://www.w3.org/TR/shacl/
[ucr]: https://www.w3.org/TR/shacl-ucr/

### Announce SHACL in API Documentation

An API which chooses to use SHACL in addition to the built-in `hydra:Class` `MUST` announce that fact in its API Documentation resource

```json
{
"@context": {
"hydra": "http://www.w3.org/ns/hydra/core#",
"requires": { "@id": "hydra:requires", "@type": "@id" }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather reuse hydra:required.

},
"@id": "/api",
"@type": "hydra:ApiDocumentation",
"requires": "http://www.w3.org/ns/hydra/shacl"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong opinion on the hydra:requires term and its semantics.

Alternatively I thought hydra:uses or hydra:imports

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think better approach would be just to point to the SHACL's namespace, http://www.w3.org/ns/shacl#

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe 🤔

As seen below, my intention was for http://www.w3.org/ns/hydra/shacl to mean "SHACL extensions" and itself would become a namespace for additional terms

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I remember now. I was thinking that maybe the requirements could be more fine-grained like

<> hydra:requires hashi:OperationExpects, hashi:IriTemplate

But coming back to my first answer still. The meaning is that "the API requires SHACL extension". Which is slightly more specific than just SHACL

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced it's good direction. I think hydra should be extended in places where it does not provide any tools - replacing them is not a good move. It's better to create a separate spec.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think hydra should be extended in places where it does not provide any tools - replacing them is not a good move

What do you mean by that? I do propose a separate spec, auxiliary to core

-http://www.w3.org/ns/hydra/core#
+http://www.w3.org/ns/hydra/shacl#

Copy link
Member

@alien-mcl alien-mcl Jun 19, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like an idea where terms built in hydra would be replaced with some custom ones. In your example, hashi:IriTemplate would probably replace hydra:IriTemplate. I think it's not a good approach

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, no, I do not propose to replace anything.

I only wanted to propose a more fine-grained choice of which parts of the API are described with SHACL. hashi:IriTemplate indeed may have been an unfortunate choice

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hashi:IriTemplate indeed may have been an unfortunate choice

Indeed - unfortunate. Could we come with a better one?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh, is that really necessary? It's not even mentioned in the pull request.

For the time being let's assume that only HASHI as a whole can be required, imported or used. However we choose.

}
```

By adding the `requires` property with the `hydra/shacl` identifier the client will be prepared to find SHACL terms within the payloads coming from this API.

### Describing request bodies

The most obvious use for Shapes is to use them to annotate supported operations' payloads. They will simply be added alongside `hydra:Class` as objects of `hydra:expects`.

```json
{
"@context": "/api/context.jsonld",
"@id": "/api",
"@type": "hydra:ApiDocumentation",
"supportedClass": [{
"@id": "mov:Movies",
"supportedOperation": {
"@type": "schema:CreateAction",
"method": "POST",
"expects": ["mov:Movie", "mov:NewMovieShape"]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeps "mov:Movie" to keep backwards compatibility

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part is kind of tricky. What does the set means: is it a union or alternative?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Union, obviously. Nothing tricky. Read it like:

The operation expects mov:Movie which should conform to mov:NewMovieShape

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this very case it may be obvious, but in general it's not. Imagine a case when an operation expects i.e. good or service (which are in many cases disjoined classes). Intent was to tell the client that the operation (basket?) expects a resource of one of those types - resource being both of the classes would violate the vocabulary)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see how this is relevant. Hydra Core would need to provide a way for alternatives just like OWL and SHACL do. Otherwise a set of objects is a union just like I would expect it anywhere else

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's not directly related to possible extensions, but I expect questions in this (and similar) situations.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise a set of objects is a union just like I would expect it anywhere else

It's not that straight forward. From pure RDF perspective it's a set of values and there is nothing supporting a statement a resource should be of all of the provided values. I'd actually prefer 'one of the possible values' rather than 'all of them'. I believe we need to specify it in the spec to avoid discrepancies.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't make me look up deep in some RDF specs but I'm pretty sure that the semantics would be precisely that. If you have multiple objects of a property, then they form a set.

This is how OWL works and also how SHACL works. Hence the provide owl:oneOf and sh:or respectively to explicitly describe alternative choices without ambiguity.

If that is not enough, let's look at rdf:type. <Karol> rdf:type schema:Person, foaf:Person means that that resource has both of those types, not one of the possible values. Trying to apply different semantics to expects would IMO violate the principle of least astonishment.

}
}, {
"@id": "mov:Movie",
"@type": "Class",
"supportedProperty": [{
"property": "schema:name"
}, {
"property": "mov:director"
}, {
"property": "schema:genre"
}]
}]
}
```

By adding a second value to the `expects` annotation, the client will discover that the payload of this operation is expected to be not only a representation of a `mov:Movie` but also that the request body has to conform to the given SHACL Shape.

{% hint style="info" %}
Client which do not understand SHACL will ignore it and proceed with only the `hydra:Class` description.
{% endhint %}

For clarity the `mov:NewMovieShape` is presented separately as it would appear in the API Documentation resource or as its own individual resource.

```json
{
"@id": "mov:NewMovieShape",
"@type": "sh:NodeShape",
"sh:targetClass": "mov:Movie",
"sh:property": [{
"sh:path": "schema:name",
"sh:or": [{
"sh:datatype": "xsd:string"
}, {
"sh:datatype": "rdf:langString"
}],
"sh:name": "Title",
"sh:description": "Movie's title",
"sh:minCount": 1
}, {
"sh:path": "mov:director",
"sh:class": "schema:Person",
"sh:nodeKind": "sh:IRI",
"sh:maxCount": 1
}, {
"sh:path": "schema:genre",
"sh:in": [
"Comedy",
"Drama",
"Documentary"
]
}]
}
```

While the Shape repeats some of the information typically found on the supported class `mov:Movie` it is clear that it can also provide much more information about the expected payload of the request to create a movie resource, which would be otherwise impossible with just Hydra Core terms:

1. About `schema:name` property:
* At least one value is required
* There can be multiple values
* Values can be a plain string of string with language tag
2. About `mov:director` property:
* Must be a resource identifier of a `schema:Person` resource
* There can be at most one value
3. About `schema:genre` property:
* Values should be one of those provided by the `sh:in` enumeration

{% hint style='info' %}
The only gray area is the director property, where it is up to the client to figure out the available instances of `schema:Person`
{% endhint %}

### Describing IRI templates

Another way for a client to create HTTP requests is by filling an IRI template with string values and dereferencing the resulting identifier. This works well for example for filtering a collection with query strings.

An instance of tht `mov:Movies` class above could provide such a search template to filter movies by genre.

```json
{
"@id": "movies",
"@type": ["Collection", "mov:Movies"],
"search": {
"@type": "IriTemplate",
"template": "movies{?genre}",
"mapping": [{
"@type": "IriTemplateMapping",
"variable": "slug",
"property": "schema:genre"
}]
}
}
```

While this is enough to match an existing resource which has the `schema:genre` property, it is too little information to gather user input, such as with a form on a web page.

To enrich the template definition, a SHACL shape can be added to further specify the template mappings.

```diff
{
"@id": "movies",
"@type": ["Collection", "mov:Movies"],
"search": {
"@type": "IriTemplate",
"template": "movies{?genre}",
"mapping": [{
"@type": "IriTemplateMapping",
"variable": "slug",
"property": "schema:genre"
- }]
+ }],
+ "hashi:shape": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this relation. Resource <movies> is in relation <hashi:shape> with some blank node resource. How does it tell the client that values for schema:genre can be used from the given set? What if schema:genre defines already a set somewhere else (i.e. API documentation).

Copy link
Collaborator Author

@tpluscode tpluscode Jun 17, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resource <movies> is in relation <hashi:shape>

Did I get the formatting wrong? It's the IriTemplate which has that relation

some blank node resource

sh:Shape 🙄

How does it tell the client that values for schema:genre can be used from the given set?

In turtle you would have

<movies> hydra:search [
  hydra:mapping [
    hydra:property schema:genre
  ] ;
  hashi:shape [
    sh:property [
      sh:path schema:genre ;
      sh:in ( "Comedy" "Drama" )
    ]
  ]
] .

To populate the template a resource is constructed which has the schema:genre property.
hydra:mapping informs the client how to map it to the template variables
hashi:shape provides information about the resource constraints

What if schema:genre defines already a set somewhere else (i.e. API documentation).

You're looking for something which is not there. I know not of an alternative way to define such a set other than SHACL. And even so, I think that those SHACL descriptions would be applied locally and trump one defined elsewhere

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that those SHACL descriptions would be applied locally and trump one defined elsewhere

I'm not sure I understand. I believe these kind of descriptions could be applied both on API documentation and resource level. I imagine situation where an API documentation says that property schema:genre in shape Movie can accept any value taken from the list. But a resource description gives another list. How those are related to each other?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, it's a little bit out of scope of this constrained example. Sure, the sh:property could somehow reference a shared list of elements, defined in the API Documentation. Sounds like a good idea.

In the first version of an actual HASHI spec I would also define a way to refer to a hydra:Collection so that the choices can be loaded from the API itself.

Either case however goes beyond what I want to propose here, which is extending the IRI Template definition with a Shape by the use of hashi:shape property

+ "@type": "sh:Shape",
+ "rdfs:label": "Search movies collection",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think rdfs:description fits better to the text provided.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite irrelevant but what would you have as the label then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something shorter I think. Just search to feed a button?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is rather minuscule

+ "sh:property": [{
+ "sh:path": "schema:genre",
+ "sh:name": "Genre",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe sh:name can be replaced with either hydra:title or just `rdfs:label'

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hydra:title has no meaning in SHACL context

sh:name is SHACL's method for annotating PropertyShape

I know it's weird but decided that way because rdfs:label would effectively annotate the Shape where the intention is to indirectly annotate the property 🤷

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but which label client should use i.e. to build an UI?

Copy link
Collaborator Author

@tpluscode tpluscode Jun 18, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of what choice? There is only one label here (sh:name) 🙂

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IriTemplate can accept hydra:title, which is prefered in context of hydra. If not provided, supported property can also have one.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would consider a Shape as separate concerns.

So, the API describes its hypermedia with a Shape. That Shape can be then passed to a component which understand SHACL to build the UI for example and run validation. That SHACL-specific component would not be aware of Hydra though, hence I stick to only terms defined in SHACL.

The Shape is its own independent description which a Hydra client can take advantage of

+ "sh:in": ["Comedy", "Drama", "Documentary"]
+ }]
+ }
}
}
```

Just like in the shape to create a new movie, the values of `schema:genre` property would be described as an enumeration of string values to present in a select box.

<img alt="select example" src="/images/search-select.png" width="400px">
Binary file added images/search-select.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions summary.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
* API Documentation
* [SHACL](api-documentation/shacl.md)
* Examples
* [Movies](movies/index.md)
* [1 API entry point](movies/entry-point.md)
Expand Down
10 changes: 0 additions & 10 deletions yarn.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -961,16 +961,6 @@ gitbook-cli@^2.3.2:
tmp "0.0.31"
user-home "2.0.0"

gitbook-plugin-forkmegithub@^2.2.0:
version "2.2.0"
resolved "https://registry.yarnpkg.com/gitbook-plugin-forkmegithub/-/gitbook-plugin-forkmegithub-2.2.0.tgz#b64b62a1402b099b1b48c396a9d5b0aba967c04a"
integrity sha1-tktioUArCZsbSMOWqdWwq6lnwEo=

gitbook-plugin-hints@^1.0.2:
version "1.0.2"
resolved "https://registry.yarnpkg.com/gitbook-plugin-hints/-/gitbook-plugin-hints-1.0.2.tgz#e6589983e3ea84749f16dc631e282fafd45ec39b"
integrity sha1-5liZg+PqhHSfFtxjHigvr9Rew5s=

github-url-from-git@~1.4.0:
version "1.4.0"
resolved "https://registry.yarnpkg.com/github-url-from-git/-/github-url-from-git-1.4.0.tgz#285e6b520819001bde128674704379e4ff03e0de"
Expand Down