REST, GraphQL, MQTT and WebSocket Interface Overview

The Management System and the Engine have APIs to access and modify the data on the respective system. Our goal is to provide the same interfaces for both systems where useful (because both systems handle the same process data). For the specific endpoints, please have a look onto the linked pages of the main wiki page.

Currently, we provide the following interfaces: HTTP(S) (REST), self-implemented GraphQL, MQTT, WebSockets. The interfaces are mapped in a consistent way between all technologies.

REST API

Available on the following Systems: Engine, Management System

The REST API is provided on the Engine and MS
The API interface description is separated content-wise to multiple wiki pages. See:
The MS uses it mainly for the Frontend to receive the data
The MS has an OpenAPI documentation published on the website about the API: https://docs.proceed-labs.org/msapi.html

Authentification

Currently, only the Management System in the production version requires a (user) authentification for REST API calls

TODO: Descibe the Auth System for REST API calls

GraphQL

Available on the following Systems: Engine

The Engine enables an self-implemented version of an GraphQL interface on all of our REST APIs.

Requests to the Engine pass through a middleware that allows for response objects to be filtered on the Engine before they are sent back to the requesting client - resulting in kind of an GraphQL API. The attributes that are to be sent back are specified using the entries query parameter.

If the complete response would look like the following:

{
  "processId": "<process-id>",
  "processVersion": <number: date-time in ms since 1.1.1970>,
  "processInstanceId": "<random uuid>",
  "globalStartTime": <number: date-time in ms since 1.1.1970>,
  ...
  "tokens": [{
      "tokenId": "<random 7 char ID with numbers and letters>",
      "currentFlowElementId": "<flow-element-id>",
      "currentFlowElementStartTime": <number: date-time in ms since 1.1.1970>,
      "localStartTime": <number: date-time in ms since 1.1.1970>,
      "localExecutionTime": <number: time in ms>,
      "currentFlowNodeProgress": <number: 0-100>,
      ...
    }
  ],
  "variables": {},
  "log": [
      {
        "tokenId": "_safdss...",
        "flowElementId": "<flow-node-id>",
        "machine": { "id": "<machine-id>", "ip": "<ip>", "name": "<machine-name>"},
        "startTime": <number: date-time in ms since 1.1.1970>,
        "endTime": <number: date-time in ms since 1.1.1970>,
        ...
      },
  ],
  ...
}

Then adding the query ?entries=processId,processInstanceId,log to the request URL will return:

{
  "processId": "<process-id>",
  "processInstanceId": "<random uuid>",
  "log": [
      {
        "tokenId": "_safdss...",
        "flowElementId": "<flow-node-id>",
        "machine": { "id": "<machine-id>", "ip": "<ip>", "name": "<machine-name>"},
        "startTime": <number: date-time in ms since 1.1.1970>,
        "endTime": <number: date-time in ms since 1.1.1970>,
        ...
      },
  ],
}

It is also possible to filter nested objects or objects inside arrays by defining the required attributes for the nested object in parentheses following the name of the nested object. For example: ?entries=processId,processInstanceId,tokens(tokenId,currentFlowElementId).

{
  "processId": "<process-id>",
  "processInstanceId": "<random uuid>",
  "tokens": [{
      "tokenId": "<random 7 char ID with numbers and letters>",
      "currentFlowElementId": "<flow-element-id>",
    }
  ],
}

MQTT

Available on the following Systems: Engine, Management System

If configured, an Engine sends monitoring data to topics and enables the usage of the REST API via MQTT
The Management System receives monitoring data via MQTT. Currently it does not send data to MQTT.

Enabling the MQTT Interface in the Engine

To enable MQTT messaging in the engine a serverAddress has to be set in the Engine config file

{
...
"messaging": {
  "serverAddress": <String>, // url or ip (and port) of a mqtt broker, this will enable the base MQTT functionality on the Engine
  "username": <String>, // optional, needed if the MQTT broker requires some kind of user authentication
  "password": <String>, // optional
  "baseTopic": <String>, // optional, possible topic-name which is prepended to the usual topics
}
...
}

REST API on MQTT: Mapping of the HTTP Interface to MQTT

We support the same REST API also via MQTT. Every REST path is a topic, the message contains the request method and the response is sent on the same topic:

Engine needs to subscribe to every topic, i.e. Networking path => based on the configuration it is <mqtt-server>/<base-topic>/proceed-pms/engine/<engine-id>/api/<Networking-Path>
Engine receives in this topic the request data, e.g. {"type": "request", "id": "<short-id>", "method": "POST", "body": {...}, "query": {<query-object>}, "page": <number> }
Engine sends response to same topic: {"type": "response", "id": "<short-id>", "statusCode": <number>, "body": {...}, "page": <number>, "pageCount": <number>, "statusCode": <number> }
If there is some error the response differs: {"type": "response", "id": "<short-id>", error: <string>, "statusCode": <number>}

=> id is necessary and makes the request unique, because multiple requests at the same time are possible and without an id the response could not be mapped to a request

Hint: the REST API is only available for MQTT Engine configuration and not for process level configuration.

Monitoring data: publishing of Runtime Data

The Engine can continuously publish data about the process executions to an MQTT topic for monitoring purposes. There are two options to configure this: at the Engine configuration level (global for all processes) and inside a process definition (only for one process and all its instances).

Engine Configuration level

If MQTT is configured at Engine level, the monitoring data will be sent for all processes and all instances that are running on the Engine.

BPMN Configuration Level

MQTT can be configured inside the meta-data of a BPMN process:

In BPMN XML it serializes like this:

The monitoring data will be sent for all instances of the specific BPMN process and not for any other processes. The data is only valid for the runtime of the process instance.

Transmitted Monitoring Data and their Topics

After configuring MQTT on the Engine or process level, the following monitoring data is sent to the configured <mqtt-server> or <BPMN-MQTT-Server-URL> server:

Engine Status:

content: the status of the PROCEED Engine that it is alive
has Last Will Message activated: if Engine goes down, then running is set to false
For Engine Level Configuration on topic: <base-topic>/proceed-pms/engine/<engine-id>/status
For Process Level Configuration on topic: <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/instance/<instance-id>/engine/<engine-id>/status

{
"running": <Boolean>,
"version": <String> //version number of the package.json file from the compiled Engine
}

Machine Monitoring Information:

content: the monitoring data of the Machine that the PROCEED Engine is running on (see Machine Module and REST API Endpoint)
sends the data to the MQTT topic every engine.loadInterval seconds (default 10s, see Configuration Keys)
For Engine Level Configuration on topic: <base-topic>/proceed-pms/engine/<engine-id>/machine/monitoring
For Process Level Configuration on topic: unspecified

Logging Data:

content: the logging data of the PROCEED Engine (see Logging REST API)
For Engine Level Configuration on topic
- standard (engine) and all process logs: <base-topic>/proceed-pms/engine/<engine-id>/logging
- only standard (engine) logs: <base-topic>/proceed-pms/engine/<engine-id>/logging/standard
- only all process logs: <base-topic>/proceed-pms/engine/<engine-id>/logging/process
- logs of a specific process instance: <base-topic>/proceed-pms/process/<definitions-id>/instance/<instance-id>/logging
For Process Level Configuration on topic: <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/instance/<instance-id>/logging

Process definition:

content: the process specification, its meta information and a list of included steps
the message is retained for new subscriptions
For Engine Level Configuration on two topics:
- <base-topic>/proceed-pms/engine/<engine-id>/process/<definitions-id>/version/<version-id>
- <base-topic>/proceed-pms/process/<definitions-id>/version/<version-id> -> last updated by any engine (decentralized approach)
For Process Level Configuration on topic: <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/version/<version-id>

{
processId: <String>, (aus BPMN)
processVersion: <number>, (aus BPMN)
processName: <String>, (aus BPMN)


timePlannedOccurrence: <String>, (Bsp: 2023-05-23T00:00:00+02:00, aus BPMN)
timePlannedDuration: <String>, (Bsp: P1M1DT2H2M, aus BPMN)
timePlannedEnd: <String>, (Bsp: 2023-05-25T00:00:00+02:00, aus BPMN)
costsPlanned: <String>, (aus BPMN)
orderNumber: <String>, (aus BPMN)
orderCode: <String>, (aus BPMN)
customerId: <String>, (aus BPMN)
customerName: <String>, (aus BPMN)

processSteps: [  (aus BPMN)
	{   
                stepId: <String>, 
		name: <String>, 
		taskType,  
		description, 
		milestones,  
		timePlannedOccurrence: <String>, (Bsp: 2023-05-23T00:00:00+02:00)
		timePlannedDuration: <String>, (Bsp: P1M1DT2H2M)
		timePlannedEnd: <String>, (Bsp: 2023-05-25T00:00:00+02:00)
		costsPlanned: <String>, 
		property1 .. propertyX 
        },
	{ ... step2 - stepX ...},
],

bpmnString: <String>
}

Hint: processId is actually the <definitions-id> of a BPMN file

Instance data:

content: the instance and token object of the running process (see respective wiki page)
is send every time on token change
For Engine Level Configuration on two topics:
- <base-topic>/proceed-pms/engine/<engine-id>/process/<definitions-id>/instance/<instance-id>
- <base-topic>/proceed-pms/process/<definitions-id>/instance/<instance-id> -> last updated by any engine (decentralized approach)
For Process Level Configuration on two topics:
- <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/instance/<instance-id>/engine/<engine-id>/instance/<instance-id>
- <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/instance/<instance-id> -> last updated by any engine (decentralized approach)

WebSockets

Available on the following Systems: Management System

A WebSocket API is available on the Management System. This is used for editing and changing process diagrams collaboratively.

For editing a process, the BPMN is loaded normally via the REST interface. Afterwards, every change a sent as event from the client to the server via WebSockets. On the server, the change is distributed to all other clients that has the process loaded, so that every client can built a correct version of the current process diagram.

%%TODO: explain WebSocket interface (see source code)

This is the Dev Wiki

Provide feedback

Saved searches

Use saved searches to filter your results more quickly