Templates and data access

AppBase: Templates and Data Access

Appbase provides an abstraction and set of conventions for access to RDF data sources to make them easier to present via a scripting templates.

For information on configuring a velocity engine to render templates see Configuration.

Template environment

When a template is invoked a set of context variables will be prebound, these are:

variable	binding
$request	the servlet request object
$response	the servlet response object
$root	the root context path for the servlet
$lib	a java utility library (instance of Lib) which in turn can be extended by configuring components which are instances of LibPlugin and registering them using the plugin or plugins configuration property of the velocity renderer component
$app	the App which owns this renderer
all query parameter names	first string value to which they are bound in the request URL
all component names in app	the corresponding component; for example, if a component source (an instance of WSource) is defined in the configuration file then $source will be bound in the velocity environment so that calls like $source.select and $source.search can be used.

Prefix handling

One of the pains of writing templates for RDF UIs is handling URIs and prefixes. After several problems caused by models not having the prefix mappings they were expected to have I've converged on a brute force approach to prefix handling ...

Each app has a single PrefixService defined which provides prefix mapping. That mapping is used:

• to expand property/resource curies in templates
• to expand SPARQL queries with relevant prefix declarations
• to provide short id references for names in the data (cube) API (TBD)
• to define a JSONLD context for JSONLD rendering (TBD)

To configure such a service include something like the following in your configuration file:

prefixes             = com.epimorphics.appbase.core.PrefixService
prefixes.prefixFile  = {webapp}/WEB-INF/prefixes.ttl

There is a built-in default set of prefixes which will be used instead if no such PrefixService is explicitly defined. If a PrefixService is defined then these default prefixes will be added to its values. The defaults are:

@prefix rdf:   <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .
@prefix owl:   <http://www.w3.org/2002/07/owl#> .
@prefix xsd:   <http://www.w3.org/2001/XMLSchema#> .
@prefix dct:   <http://purl.org/dc/terms/> .
@prefix skos:  <http://www.w3.org/2004/02/skos/core#> .
@prefix foaf:  <http://xmlns.com/foaf/0.1/> .
@prefix jtext: <http://jena.apache.org/text#>

This gives a predictable prefix environment which the appbase internals can rely on.

SparqlSource

The core internal abstraction for RDF data sources is SparqlSource which provides methods for running select, describe and construct queries.

Implementations of SparqlSource currently provide for memory based datasets, TDB datasets and remote SPARQL endpoints (e.g. Fuseki).

ssource              = com.epimorphics.appbase.data.impl.FileSparqlSource
ssource.files        = load
ssource.textIndex    = default

configures a memory-based dataset which is initialized by reading files at app startup. The files value can be a comma separated list of file names, all of which will be loaded into a single combined model. If any file name is a directory then all RDF files (files with suffix ttl, rdf, owl, nt, n3 or xml) in that directory will be loaded.

The textIndex parameter is optional. If present then an in-memory Lucene index will be built to support jena-text free text queries. If its value is "default" then a text index over rdfs:label will be created. Otherwise it should be a comma-separated list of properties to be indexed (which can be curies using the app's prefix settings).

rsource              = com.epimorphics.appbase.data.impl.RemoteSparqlSource
rsource.endpoint     = http://localhost:3030/test/query

configures access to a remote SPARQL endpoint at the given URL.

tsource              = com.epimorphics.appbase.data.impl.TDBSparqlSource
tsource.location     = /home/der/projects/data/test/tdb
tsource.index        = /home/der/projects/data/test/text

configures a local TDB data store with optional jena-text index.

Update access to SparqlSources

SparqlSources also support update (update(UpdateRequest)) if isUpdateable() is true.

A DatasetAccessor for graph level update may be available through getAccessor() but read-only sources will return null for this.

The update handler and/or accessor will perform the necessary wrapping to handle the type of sources. So for in-memory sources locking is done, for TDB sources the accessor supports transactions, for remote sources a remote accessor or used.

Wrapped data source: WSource

For use in UI templates the SparqlSource is normally wrapped by an instance of WSource which provides scripting-friendly convenience functions.

To configure a wrapped source use something like:

source = com.epimorphics.appbase.data.WSource
source.cacheSize = 1000
source.source = $ssource

The cacheSize setting is optional and controls the number of resource descriptions that will be cached locally.

Aside on description caching

It is common in UIs to want to want to be able to render a bunch of resources using at least their labels but potentially other properties. The choice of label property to use typically involves selecting from a range of properties (skos:prefLabel, skos:altLabel, rdfs:label, dct:title, foaf:name) and selecting a matching language from a set of label values. When the data source is remote the cost of requesting such label, or broader, descriptions is high.

Appbase handles this by catching partial or complete resource descriptions and provides a means to request appropriate descriptions in batches as well as making individual describe/select requests on demand.

WSource key methods

For many methods we need to provide some RDF Node as an argument. The wrapped interfaces allow you to specify such arguments using an instance of Node, RDFNode, a wrapped node (WNode) or a uri/curie string. In particular in templates we frequently use curie strings, which in turn rely on the configured prefixes. We use spec in the tables below to indicate where this flexibility is allowed.

Method	Explanation
select(query)	Run a sparql select query returning a wrapped result set. The query will be expanded to include all relevant prefixes from the app's prefixes
select(query, var1, spec1, ...)	As select but prebinds variables. The argument list should be a set of alternating variable name and value pairs
find(query, var)	Runs a select query, extracts all values for the given var and returns a list of wrapped nodes for those values
getNode(spec)	Return a wrapped node
describe(set) label(set)	Takes a list of wrapped nodes or a wrapped result set and returns a version of it in which every resource has an associated description. A describe call fetches a full sparql DESCRIBE whereas a label call just fetches a set of potential label properties.
search(query [, limit])	Free text search which returns a list of wrapped nodes which match the query, assumes the source supports jena-text.

Wrapped result set: WResultSet

Similar to an ARQ ResultSet but where all the values are wrapped nodes suitable for further template access.

Method	Explanation
copy()	Materializes the entire result set in memory as wrapped nodes. The resulting WResultSet can iterated over multiple times whereas the initial WResultSet will materialize wrapped results on demand and will only allow one complete iterable iterator over the set of results.
getResultVars()	List of variable names in the query

A WResultSet is also Iterable which makes it easy to do #foreach loops over the results.

Wrapped result row: WQuerySolution

Method	Explanation
get(varname)	Return the value for the given variable in this row of the result set

Together these make it easy to issue and loop over a sparql query in a velocity template. For example:

#foreach($row in $source.select("SELECT ?i WHERE {?i a ?c}", "c", $uri) )
  <tr>
    <td>#render($row.i)</td>
  </tr>
#end

Here we take a URI $uri for a class (perhaps passed as an query parameter to the template page) and list instances of that class in some given wrapped source $source.

If the #render macro needs access the labels of the instances then it may be more efficient to request these in a batch by changing the first line to:

#foreach($row in $source.label( $source.select("SELECT ?i WHERE {?i a ?c}", "c", $uri) ))

Wrapped RDF node: WNode

This wraps up a single RDF Node (actually a Node) value. In the case of a resource node it may have an associated description graph which may contain a complete description (result of SPARQL DESCRIBE) or a partial description just containing label values (skos:prefLabel, skos:altLabel, rdfs:label, dct:title, foaf:name). It also has a reference to an associated wrapped source. Requests for labels or other properties of a resource will consult the attached description if present and sufficiently complete, otherwise they will provide an additional query to source. In this way it is possible preload descriptions in a batch call for efficiency, and to cache them across calls, but the API will function whether the descriptions have been preloaded or not.

Method	Explanation
isResource() isURIResource() isLiteral() isAnon()	Tests if the node is a resource, uri resource, literal or blank node
isNumber()	Tests if the node is a literal corresponding to some number (integer, float, decimal)
asNumber() asInt() asFloat()	For a number literal returns its value as an Number, integer or float, throws an exception otherwise
asLiteral()	Returns the node as a Literal, or null if it is not a literal node
getLiteralValue()	For a literal returns the corresponding java object value
getDatatype()	Return the URI for the datatype of a literal, after prefix shortening, e.g. "xsd:string"
isList()	Tests if the node is a resource corresponding to a RDF list
asList()	Returns the members of the list as a plain java.util.List of wrapped nodes
getURI()	Returns the URI of a URI resource
getLabel() getLabel(language)	Return a suitable display label for the node. For a literal node this is just its lexical form. For a resource node it is picks the first value of an ordered list of possible label properties (skos:prefLabel, skos:altLabel, rdfs:label, dct:title, foaf:name). The second call picks the version of the label which corresponds to the given language code if any, otherwise picks a plain literal or whatever language is available.
getPropertyValue(spec)	Return a value of the given property on the node as another wrapped node.
listPropertyValues(spec)	Return all values of the given property on the node as java.util.List of wrapped nodes.
hasResourceValue(spec-prop, spec-value)	Tests if the node has a value for the given property which is the given resource
listProperties()	Return a list of all property values for the node. Each entry in the list will be a PropertyValue object containing the property as a wrapped node (getProp()) and its values as a list of wrapped nodes (getValues()).
listInLinks(spec)	Return a list of all the wrapped nodes which connected to this one via the specified property
listInLinks()	Return all nodes which connection to this one via any property, return is a list of PropertyValue objects, but where the property is being read as inverted.
connectedNodes(path)	Return a list of all the wrapped nodes which are connected to this via the given sparql property path expression (which can use prefixes from the app's prefix configuration

All of the calls that involve access to properties of the node will try to use the node's attached description if it exists but otherwise will trigger a new query to the source.

Lib

The facilities provided by the library object are:

Method	Explanation
escapeHtml(string[,limit])	Escape a string for embedded HTML characters, optionally limiting it to a maximum character count
regex(string,regex)	If the regex matches returns an array of all the capture group values, otherwise null
pathEncode(string)	Encode a string so it can be safely send as a query parameter or URL path parameter. Uses just %-encoding.
pathDecode(string)	Reverse the pathEncode encoding (not normally needed because query/path parameters will be decoded on reception by the web app container

There are other methods currently lying around on Lib that may be removed or improved in the future.

In addition if a LibPlugin component is configured then it can be accessed through the library. For example, the configuration:

myplugin = com.epimorphics.appbase.webapi.ExampleLibPlugin

velocity = com.epimorphics.appbase.templates.VelocityRender
# Other velocity configuration goes here
velocity.plugin = $myplugin

Will mean that the velocity context variable $lib.myplugin refers to an instance of ExampleLibPlugin.