Initial engine support for submissions

[eyelidlessness]

I have verified this PR works in these browsers (latest versions):

• Chrome
• Firefox
• Safari (macOS)
• Safari (iOS)
• Not applicable

What else has been done to verify that this works as intended?

• Quite a bit of new scenario testing
• Wired up display of serialized submission XML in ui-solid's debug view (replacing the slightly broken version of it that was there before)

Why is this the best possible solution? Were any other approaches considered?

This is largely consistent with the design in #188, so from a broad design perspective this is mostly answered there.

Several other implementation approaches were considered, which I'll detail along with the description of the actual approach I landed on.

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

When I first approached testing the XML serialization aspect of this change, and specifically it being client reactive, I had originally wanted to test those at the xforms-engine level. I backed out of that when I found that testing behavior with repeats surfaced the dreaded InconsistentChildrenStateError. In theory this suggests that we could encounter similar in downstream client-reactive situations. I was not able to reproduce the issue in any of:

• scenario
• ui-solid
• web-forms (not updated here, but I did a quick integration locally to verify)

Do we need any specific form for testing your changes? If so, please attach one.

N/A.

What's changed

Engine/client API

This is more or less consistent with the design in #188, with the following deviations:

1. I did go with some of the changes discussed in my later comments there. I don't think we explicitly folded them into the agreed design, but I don't think they'll be controversial either.

2. Additive: I also introduced a node-level submissionState property, which contains a client-reactive submissionXML getter.

Client-reactive submissionState

The submissionState approach serves a few purposes:

• Ensure we stay mindful that we may want submission state to be explicitly client reactive in general

• Provide a clear signal when/where a future change is incompatible with that goal

• Establish node-level semantics for serialization, appropriate for each node type's hierarchical structure

• Allow for custom node-level serialization logic (possibly applicable: mapping submission attachments to unique file names; configurable omission logic for non-relevant repeat instances)

• As is done in ui-solid, support inspection/debugging use cases

We may want to consider folding the submissionXML property into each node's regular currentState object. I think it would be appropriate, but I didn't want to burden this change with the exploratory work around how we'd mix derived state into its source object (i.e. submissionXML is derived from each node's currentState.children or currentState.value).

Engine internal

I don't think there's really much to say here. The bulk of the work is serialization, which is relatively straightforward since our in memory representation maps almost exactly to our serialized representation. Some new internal interfaces were added to support this, but they're largely reflective of what was already in place apart from the new API surface area.

Partial/incomplete functionality

Since we do not currently support uploads/submission attachments, there was little pertinent work to do on the chunked option. There is very minimal stubbed support, largely for the minor difference in shape of SubmissionResult<'chunked'>. There is currently no committed work on these fronts:

• Actual chunking of attachments
• Validation of form state against a configured maximum chunk size
• Mapping of attachments to unique file names

While it's possible to implement these and validate them with automated testing, it would be mostly speculative until we have actual File data to submit.

I did explore all of these, and will be able to reference them from stashes. But I felt it best not to put that much speculative work into functionality which is already ready for real use.

Alternative serialization approaches

I explored several alternative approaches to serialization. These are the most notable.

• Creating an engine-internal "mini client", demonstrating that the serialization is effectively possible with existing client APIs. I think there would be other value in this approach, and it could also serve as a home for other hypothetical "possible for a client, valuable from the engine" functionality. Ultimately I dropped the effort when I encountered annoying tooling yak shaves, specifically around TypeScript configuration (it's really not designed for creating an artificial package boundary with cyclic dependencies, which is what the approach requires).

• I had discussed specific approaches to mapping attachments to unique file names. The serialization implementation I landed on will need to change to support that use case: it involved a provision for injecting the mapping logic at the serialization call site, which is not consistent with this implementation's use of getters. I don't think the change will be significant, but as explained above I wanted to avoid over-engineering so much speculative, out of scope functionality.

Scenario

With the initial serialization work complete, it was trivial to observe the one existing submission test ported from JavaRosa passing. I also added a slew of tests in the following areas:

• Serialization of each hierarchical consideration:
  • Leaf nodes and their values
  • Ancestor nodes and their descendants
  • Repeat ranges and their non-element-ness
• Serialization of repeat instances, specifically
• Omission of non-relevant nodes from serialization

Moreover I added tests exercising the (I think) novel aspects of our design in #188:

• status: 'pending', i.e. production of serialized invalid form state, accompanied by all validity violations present at that time
• status: 'ready' to explicitly differentiate valid form state ready for submission

I'm on the fence about how much to accommodate the incomplete chunking support at a test level. I would really like to have something in place which will fail when we add that support, but I think that would again require out of scope speculative work to have any value.

[changeset-bot]

🦋 Changeset detected

Latest commit: 8edf375

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
@getodk/xforms-engine	Minor
@getodk/scenario	Minor
@getodk/ui-solid	Patch
@getodk/common	Patch
@getodk/xpath	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR