Incremental Updates for batch interface using ETags
When querying the discovery resource with the batch interface, the response can become extensive because it contains the payloads of selected discoverable and accessible resources. To optimize this process and reduce unnecessary data transfer, we can leverage the ETag CoAP option. By utilizing ETags, we can avoid resending the entire payload if we have previously requested it and no changes have occurred. However, it's important to note that once at least one relevant resource has been updated, created, or deleted, the previous ETag becomes invalid, and we will receive the full payload again. This behavior aligns with the specifications outlined in RFC 7252.
We have extended the functionality of ETags to further enhance efficiency by avoiding the transmission of redundant data whenever possible. This optimization relies on specific properties of the IoTivity ETag implementation:
- All resource ETags are unique.
- ETags are strictly increasing, meaning that the later a resource is created or updated, the higher its ETag value will be.
- ETags are regenerated on device reset, potentially altering the order of resource ETags.
When the device has not undergone a reset, we can employ a previous ETag value to determine which resources have changed since that ETag was generated. In essence, all resources with ETag values higher than the previous ETag have been updated or created. To implement this mechanism, we can follow this algorithm:
- The device receives a list of ETags.
- Iterate through the received ETags and identify the maximum ETag value for which a corresponding resource exists on the device.
- If at least one such ETag exists in the list, we conclude that the device has not been reset.
- When generating a batch response payload, omit the payloads of resources that possess an ETag value equal to or lower than the maximum ETag. These resources have already been provided to the user, and their contents remain valid.
While RFC 7252 already allows the CoAP request to contain multiple ETags, the behavior differs from the one specified for incremental updates above. To avoid breaking the standard behavior, we utilize the query options of a CoAP request.
When the incChanges key is found in the query, the value for the key is expected to be a comma-separated list of ETags encoded in URL-safe base64 without padding (see RFC 4648, 5. Base 64 Encoding with URL and Filename Safe Alphabet). It's possible to provide multiple incChanges key-value pairs in the CoAP query options, and all will be examined. All valid ETags found in the CoAP Query and ETag options will be used as input for the algorithm.
- Implement incremental updates for collections.
