diff --git a/content/device-integration/mqtt-service-bundle/faq.md b/content/device-integration/mqtt-service-bundle/faq.md new file mode 100644 index 0000000000..e61cad4bae --- /dev/null +++ b/content/device-integration/mqtt-service-bundle/faq.md @@ -0,0 +1,16 @@ +--- +weight: 90 +layout: redirect +title: FAQ +--- + +Q: How to obtain device credential?
+A: MQTT Service does not support a device bootstrap process yet. Instead, follow the [Integration life cycle](/device-integration/mqtt/#integration-life-cycle) +to bootstrap the device and obtain device credentials. Once the device credentials are obtained, ensure that they have `Mqtt Service` `ADMIN` permission. +You can achieve this for all devices by granting this permission to the `devices` global role. + +Q: Does MQTT Service support the SmartREST 2.0 protocol?
+A: Not yet, support for SmartREST 2.0 will be added in the future. + +Q: Why does MQTT Service not use standard MQTT ports 1883 and 8883?
+A: Those ports are already used by {{< product-c8y-iot >}} MQTT. Both endpoints are working together MQTT Service must use different ports. \ No newline at end of file diff --git a/content/device-integration/mqtt-service-bundle/implementation.md b/content/device-integration/mqtt-service-bundle/implementation.md new file mode 100644 index 0000000000..b60f46acf7 --- /dev/null +++ b/content/device-integration/mqtt-service-bundle/implementation.md @@ -0,0 +1,178 @@ +--- +weight: 20 +title: MQTT protocol implementation +layout: redirect +--- + +This section lists the implementation details for the MQTT Service. The MQTT Service implementation supports MQTT Version 3.1.1, support for 5.0 is planned. + +### Connecting via MQTT {#connecting-via-mqtt} + +MQTT Service is supported via TCP. Use your tenant domain as the URL. + +Available ports: + +|   | TCP | +|:-----|:----| +| TLS | 9883 | +| no TLS | 2883 | + +Port 9883 is enabled by default. It currently supports one-way SSL meaning that only the client validates the server certificate to ensure its identity. +The client is authenticated by the server via standard username and password credentials. +To enable port 2883 please contact [Product support](/additional-resources/contacting-support/). + +### Topic {#topic} + +MQTT Service topics are mapped to the Messaging Service subscriptions with identical names, including additional URL encoding. +The Messaging Service subscriptions reliably store the topic messages for asynchronous processing. +The messages stored on these subscriptions can be consumed using a dedicated [Java Client](/device-integration/mqtt-service#java-client). + +#### Topic restrictions {#topic-restrictions} + +MQTT Service does not impose any topic structure. There are just a few topic names which are reserved for historic purposes and future use, namely: +* All [SmartREST 2.0](/smartrest/smartrest-two) related topics +* `error` +* `devicecontrol/notifications` + +Other than that you are free to use any topic name which is compatible with the [MQTT specification](http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718106). + +{{< c8y-admon-info >}} +Wildcard topics (`+`, `#`) and system topics starting with `$` are not supported. +{{< /c8y-admon-info >}} + +#### Topic limit {#topic-limit} + +MQTT Service has the ability to limit the total number of topics that a single tenant can create. The current default is no limit. +When the creation of a new topic, either by creating it via the client publishing a message or subscribing to a non-existent topic, would breach the topic limit +the delivery of the packet is prevented. + +The different MQTT protocols provide the following feedback. + +MQTT 5 clients: + +* Have access to the reason code and reason string describing the failure when using QoS 1 with acknowledgements, +reason code being `QUOTA_EXCEEDED: 0x97`. + +MQTT 3.1 and 3.1.1 clients: + +* Clients only have access to the reason code describing the failure when using QoS 1 with acknowledgements and only +for the SUBSCRIBE packets, where the reason code is `0x80`. +* For the PUBLISH packets, the client will be disconnected with no further information as per the MQTT specification. + +#### Error Topic {#error-topic} + +MQTT Service provides clients the ability to review errors through messages received by subscribing to the error topic, `$debug/$error`. +When subscribing to the topic it will act as a per-client topic, meaning the client will only receive messages exclusively related to their client ID. For example, +if a client was attempting to subscribe to a new topic, and the creation of the topic would exceed the topic limit, only that client would receive an error. + +According to the MQTT 3.1.1 specification, if either the server or the client encounters a protocol violation, it must close the network connection on +which it received the control packet which caused the violation. + +In such instances MQTT clients must reconnect to be able to receive error messages from the error topic via the subscription. Error messages received after this reconnection +are from the previous session. This can lead to confusion when attempting corrective actions. Therefore, we highly recommend you to build a microservice which uses +the MQTT Service SDK to consume error messages, or use MQTT 5 for clients and make use of the reason codes feature. + +#### Topic cleanup {#topic-cleanup} + +The MQTT service will automatically remove topics which are no longer active. Topics are recognized as inactive when there are no subscriptions and +the internal publisher to the topic is closed. The publisher is responsible for publishing the modified MQTT service messages to the correct topic. +The publishers live within a cache, where the publisher expires after one hour. Due to this it can take up to an hour after removing all subscriptions from a topic +for it to be automatically deleted. + +### Payload {#payload} + +The original MQTT messages are re-packed into MQTT Service message format which includes the original payload and additional metadata fields. +Assuming Java types, the packed message structure looks as follows: + +`MqttServiceMessage` +| Field name | Type | Description | +|:-----------|:--------------------|:-------------------------------| +| payload | byte[] | MQTT payload | +| metadata | MqttServiceMetadata | Metadata from the MQTT message | + +`MqttServiceMetadata` +| Field name | Type | Description | +|:-----------------------|:--------|:------------------------------------------------------------------------| +| clientId | String | Unique MQTT client identifier, usually used as an external identifier | +| messageId | int | Unique MQTT message ID per client, available only with QoS 1 and 2 | +| dupFlag | boolean | Indicates this message is a resend by the MQTT client | +| userProperties | Map | Reserved for future use of MQTT 5.0 features | +| payloadFormatIndicator | enum | Reserved for future use of MQTT 5.0 features | +| contentType | String | Reserved for future use of MQTT 5.0 features | +| correlationData | byte[] | Reserved for future use of MQTT 5.0 features | +| responseTopic | String | Reserved for future use of MQTT 5.0 features | +| topic | String | The name of the MQTT topic that the message was published by the client | + +The [Java Client](/device-integration/mqtt-service#java-client) contains classes representing the above model. + +#### Payload restrictions {#payload-restrictions} + +MQTT Service doesn't force you to use any specific payload format. +All the incoming MQTT messages must meet the specification in terms of fixed and variable headers, but the payload for published messages is unrestricted. +Just keep in mind that you will receive exactly the same set of bytes which was sent from the device in your custom microservice +and you have to convert them to {{< product-c8y-iot >}} compatible format. + +{{< c8y-admon-info >}} +For all MQTT connections to the platform, the maximum accepted payload size is 1048576 bytes (1 MiB), which includes +both message header and body. The header size varies, but its minimum is 2 bytes. +{{< /c8y-admon-info >}} + +### Features {#features} + +#### Authentication and authorization {#authentication-and-authorization} + +Authentication types supported by MQTT Service are: + +* Username and password: The MQTT username must include the tenant ID and username in the format `/`. +* Device certificates: Not yet supported. This will be added in a future release. + +#### ClientId {#client-id} + +The **MQTT ClientID** field identifies the connected client. **ClientID** may consist of up to 128 alphanumeric characters. +Each client connecting to MQTT Service must have a unique client identifier, connecting a second client with the same identifier will result in the previous client's disconnection. + +#### Quality of Service (QoS) {#quality-of-service-qos} + +The {{< product-c8y-iot >}} implementation supports two levels of MQTT QoS: + +* QoS 0: At most once: + - The client just sends the message once (fire and forget). + - No response from the server. + - No guarantee that subscribers will receive the message. +* QoS 1: At least once: + - The client awaits server acknowledgment for each published message. + - The client should re-send the message if there was no acknowledgement from the server. + - It is guaranteed that subscribers will receive a message that was acknowledged by the server. + - Subscribers may receive more than one copy of a message. +* QoS 2: Exactly once: + - not supported + +For subscriptions, MQTT Service will deliver all messages in the QoS that the client defined when subscribing to the topic. + +#### Clean session {#clean-session} + +MQTT Service requires clean session to be set to "1" (true). We cannot guarantee that disabling clean session will work reliably, hence we recommend you to always enable clean session. + +#### Retained flag {#retained-flag} + +Retained flag is ignored. Publishing data with the retained flag on the topic is allowed but has no practical difference to sending it without the flag. + +#### Last will {#last-will} + +In MQTT, the "last will" is a message that is specified at connection time and that is executed when the client loses the connection. +Last will is fully supported by MQTT Service and like with any other publish messages you can use any unreserved topic and any payload. + +### Return codes {#return-codes} + +MQTT Service follows the MQTT specification for server responses. For example, if invalid credentials are sent in the `CONNECT` message, +the server response `CONNACK` message contains the `0x05` return code. +The return code can be treated similarly to REST API HTTP codes, such as 401. + +### MQTT 5.0 features {#mqtt-50-features} + +Support for MQTT 5.0 features will be added in the near future. + +### MQTT TLS certificates {#mqtt-tls-certificates} + +MQTT Service uses the certificates which are assigned to the main environment domain. It always sends these certificates during TLS handshake to devices. +Moreover, {{< enterprise-tenant >}}s are not able to customize those certificates via the SSL Management feature. diff --git a/content/device-integration/mqtt-service-bundle/index.html b/content/device-integration/mqtt-service-bundle/index.html new file mode 100644 index 0000000000..2c2fa984b3 --- /dev/null +++ b/content/device-integration/mqtt-service-bundle/index.html @@ -0,0 +1,4 @@ +--- +title: MQTT service +headless: true +--- \ No newline at end of file diff --git a/content/device-integration/mqtt-service-bundle/java-client.md b/content/device-integration/mqtt-service-bundle/java-client.md new file mode 100644 index 0000000000..e168acc767 --- /dev/null +++ b/content/device-integration/mqtt-service-bundle/java-client.md @@ -0,0 +1,77 @@ +--- +weight: 30 +layout: redirect +title: Java Client +--- + +The MQTT Service Java Client library provides the classes necessary to interact with MQTT Service. +The following operations are supported by the client: +* Publishing messages to the MQTT Service via WebSocket protocol. +* Subscribing to messages from the MQTT Service via WebSocket protocol + +#### Repositories and dependencies {#repositories-and-dependencies} + +Follow the [Microservice SDK](/microservice-sdk/java/#add-repositories-and-dependencies) documentation for guidance on how to configure Maven repositories. +To include MQTT Service Java Client into your project, add the following dependency inside the `` node: +```xml + + com.cumulocity.sdk.mqtt + mqtt-service-ws + ${c8y.version} + +``` + +#### Example {#example} +Example of publishing messages to the MQTT Service via WebSocket: +```java +// Message to be sent +final String payload = "Hello World"; + +// Construct a new MqttServiceMessage and set the payload +final MqttServiceMessage message = new MqttServiceMessage(); +message.setPayload(payload.getBytes()); + +// Create an instance of MqttServiceApi by specifying the server URI to connect to along with TokenApi +final MqttServiceApi mqttServiceApi = MqttServiceApi.webSocket() + .url(webSocketBaseUrl) + .tokenApi(tokenApi) + .build(); + +// Build PublisherConfig with topic to which the message is to be sent +final PublisherConfig config = PublisherConfig.publisherConfig().topic(topic).build(); + +// Build Publisher and publish MqttServiceMessage. Close the resource either by using a [try-with-resources block](https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html) as below or by calling publisher.close() explicitly +try (final Publisher publisher = mqttServiceApi.buildPublisher(config)) { + publisher.publish(message); +} catch (Exception e) { + log.error("Could not sent message to {}", topic, e); +} +mqttServiceApi.close(); +``` + +Example of subscribing to messages from the MQTT Service via WebSocket: +```java +// Create an instance of MqttServiceApi by specifying the server URI to connect to along with TokenApi +final MqttServiceApi mqttServiceApi = MqttServiceApi.webSocket() + .url(webSocketBaseUrl) + .tokenApi(tokenApi) + .build(); + +// Build SubscriberConfig with topic and subscriber name +final SubscriberConfig config = SubscriberConfig.subscriberConfig().topic(topic).subscriber(subscriberName).build(); + +// Build Subscriber +final Subscriber subscriber = mqttServiceApi.buildSubscriber(config); + +// Subscribe by passing implementation of MessageListener to handle messages from the MQTT Service. +subscriber.subscribe(new MessageListener() { + @Override + public void onMessage(MqttServiceMessage message) { + log.info("Message Received: {}", new String(message.getPayload())); + } +}); + +// Close the resources after usage +subscriber.close(); +mqttServiceApi.close(); +``` diff --git a/content/device-integration/mqtt-service-bundle/overview.md b/content/device-integration/mqtt-service-bundle/overview.md new file mode 100644 index 0000000000..72e4f23583 --- /dev/null +++ b/content/device-integration/mqtt-service-bundle/overview.md @@ -0,0 +1,64 @@ +--- +weight: 10 +layout: redirect +title: Overview +--- + +{{< c8y-admon-preview >}} +This feature is in Private Preview, that is, it is not enabled by default and maybe subject to change in the future. +Please reach out to the [product support](/additional-resources/contacting-support/) if you are interested in this feature. +{{< /c8y-admon-preview >}} + +{{< c8y-admon-req >}} +To work with the MQTT Service, the following requirements have to be met: +* The {{< product-c8y-iot >}} Messaging Service must be available on your {{< product-c8y-iot >}} platform. +* MQTT Service must be enabled for your tenant. +{{< /c8y-admon-req >}} + +MQTT Service is the new MQTT endpoint implementation of {{< product-c8y-iot >}} which provides the following benefits: + +* Support for publishing and subscribing arbitrary payloads on any MQTT topic. The topics that are used by the existing {{< product-c8y-iot >}} MQTT implementation are not permitted for use with MQTT Service. +* Support for subscribing to the data stream from MQTT Service and mapping it to {{< product-c8y-iot >}} compatible one or routing them to a different service. +* Multi-tenancy support: A single endpoint serves multiple tenants. +* Full horizontal scalability. + +This section does not describe the basics of MQTT communication. If you are unfamiliar with MQTT, we recommend +you to consult one of the numerous introductions on the internet. Some references can be found on the [MQTT website](https://mqtt.org/mqtt-specification/). + +### Architecture {#architecture} + +MQTT Service works together with the Messaging Service to provide a framework for highly customizable and flexible MQTT message processing solutions. +The diagram below illustrates how a message flows, starting from the device, through the Messaging Service, +then to the microservice where it is converted to the final {{< product-c8y-iot >}} REST request. + +![MQTT Service send](/images/mqtt-service/mqtt-service-send.svg) + +All MQTT messages coming to MQTT Service are forwarded to the Messaging Service, where they are persisted, waiting to be consumed. +A custom microservice that understands the topic and payload structure can, with the help of [Java Client](/device-integration/mqtt-service#java-client), +consume the MQTT messages, translate them to the {{< product-c8y-iot >}} format, and then use the [Microservice SDK](/microservice-sdk/java) to push them into {{< product-c8y-iot >}}. + +Similarly, messages can be sent to devices, as shown in the diagram below. + +![MQTT Service push](/images/mqtt-service/mqtt-service-push.svg) + +Like with the message coming from the device the same solution can be also applied when trying to communicate with the device. +Given the MQTT client ID and the topic, a microservice can push any MQTT message to a device via [Java Client](/device-integration/mqtt-service#java-client). + +### MQTT Service vs {{< product-c8y-iot >}} MQTT {#mqtt-service-vs-cumulocity-iot-mqtt} + +The table below presents a basic comparison between the standard {{< product-c8y-iot >}} MQTT functionality and that of MQTT Service. + +| | {{< product-c8y-iot >}} MQTT | MQTT Service | +|:-----------------------------|:--------------------------------------------------------|:----------------------------------------------------------------------------------| +| QoS | 0, 1, 2 | 0, 1 | +| Clean session | Starting with clean session is recommended | Starting with clean session is recommended | +| Retained flag | Not supported | Not supported | +| Last will | Supported | Supported | +| MQTT 5.0 features | Not supported | Support is planned | +| Authentication | Basic and device certificate | Basic authentication is supported, device certificate support is planned | +| Scalability | Horizontal | Horizontal | +| Topic format | Determined by the SmartREST 2.0 protocol | Unrestricted, SmartREST topic names are reserved and cannot currently be used | +| Payload | Determined by the SmartREST 2.0 protocol | Unrestricted, maximum message size is 1048576 bytes (1 MiB) including all headers | +| Extensibility | Limited by SmartREST 2.0 custom templates | Custom mapping microservices can support arbitrary MQTT-based protocols | +| Message processors/consumers | Build-in message processor for each SmartREST 2.0 topic | Custom mapping microservices can support multiple processors for a topic | +| JSON via MQTT | Limited feature set | Custom mapping microservices can support arbitrary JSON payloads | diff --git a/content/device-integration/mqtt-service.md b/content/device-integration/mqtt-service.md new file mode 100644 index 0000000000..b2e6311361 --- /dev/null +++ b/content/device-integration/mqtt-service.md @@ -0,0 +1,7 @@ +--- +weight: 26 +title: MQTT Service +layout: bundle +section: + - device_management +--- diff --git a/static/images/mqtt-service/mqtt-service-push.svg b/static/images/mqtt-service/mqtt-service-push.svg new file mode 100644 index 0000000000..afe1b76e9c --- /dev/null +++ b/static/images/mqtt-service/mqtt-service-push.svg @@ -0,0 +1,4 @@ + + + +
    Cumulocity
    Cumulocity
    Messaging service
    Messaging service
    MQTT Layer
    MQTT Layer
MQTT Device receiving: op/restart
MQTT Device recei...
MQTT Service
MQTT Service
Topic: op/restart
Topic: op/restart
MQTT Device receiving: op/update
MQTT Device recei...
Microservice
Microservice
Cumulocity Core
Cumulocity Core
Topic: op/update
Topic: op/update
Publisher for topic: op/restart
Publisher for top...
Publisher for topic: op/update
Publisher for top...
Maps Cumulocity
REST request to
MQTT message
Maps Cumulocity...
MQTT
MQTT
Messaging service
Messaging service
REST or Realtime API
REST or Realtime API
Text is not SVG - cannot display
\ No newline at end of file diff --git a/static/images/mqtt-service/mqtt-service-send.svg b/static/images/mqtt-service/mqtt-service-send.svg new file mode 100644 index 0000000000..10ede6744a --- /dev/null +++ b/static/images/mqtt-service/mqtt-service-send.svg @@ -0,0 +1,4 @@ + + + +
    Cumulocity
    Cumulocity
    Messaging service
    Messaging service
    MQTT Layer
    MQTT Layer
MQTT Device pushing on topic: temperature
MQTT Device pushi...
MQTT Service
MQTT Service
Topic: temperature
Topic: temperature
MQTT Device pushing on topic: pressure
MQTT Device pushi...
Microservice
Microservice
Cumulocity Core
Cumulocity Core
Topic: pressure
Topic: pressure
Subscriber for topic: temperature
Subscriber for to...
Subscriber for topic: pressure
Subscriber for to...
Maps MQTT payload
to Cumulocity
REST request
Maps MQTT payload...
MQTT
MQTT
Messaging service
Messaging service
REST
REST
Text is not SVG - cannot display
\ No newline at end of file