Add an element in the mediaFile block for Entity Lists in the OpenRosa manifest to specify a URL to get Entity information

[lognaturel]

As the implementer of a data collection client, if there are Entities in the client's representation that are not included in the server response
I want to be able to request the status of those specific Entities
So that I can delete any Entities that no longer exist on the server

As the implementer of a data collection client
I want to be able to get information about specific Entities
So that I can make sure their state is up to date in my local representation

See Collect issue getodk/collect#6231 describing desired XML API.

Strawman proposal

• Add an integrityUrl element that may only appear in a mediaFile block with type attribute (see Identify attached entity lists in the manifest response #665) set to entityList.
• The integrityUrl value is a URL that accepts a query parameter named id representing the UUID of an Entity that the client wants to know about. A comma-separated list of UUIDs is also accepted.
• The API endpoint returns an XML response as described here / Locally created entities removed from the server should eventually be removed locally collect#6231

Interesting cases to design for

Deleted and purged Entities

Currently purged Submissions are completely removed so it's possible to reuse an id. Entity purge is not yet implemented. We'll need to decide whether to keep historic UUIDs or allow reuse. Allowing reuse would only be an issue in very rare cases.

Let's not allow reuse of UUIDs when Entities are soft-deleted to avoid the case where Entities with the same UUID are in trash and active.

Rejected submissions that never resulted in Entity creation

Two concepts have emerged so far: send the instanceID to the integrity API to detect this case or keep a notion of "phantom" entities that wanted to be created but never were.

Todo

• Implement strawman
• Use in Collect
• Consider max URL size and whether query parameters are appropriate
• Consider naming of deleted element so the name matches what it represents (true: server knows of the Entity and it shouldn't be on that client; false: server either doesn't know of the Entity or it knows it and it should stay on the client)
• Post on forum and iterate on element names + response structure

CC @seadowg

[matthew-white]

• For v2024.2, the main case we need to address is soft-deleted entities, which are still present in the database and easy to look up in the entities table.
• Once we support purging entities, we'll need a way to track the UUIDs of entities that have been purged.
• We may also implement something related to submission approval, but probably not until a later release (see Tell clients to delete entities when their submissions have not been approved #685).