More form updates

[kschiffer]

Summary

This PR extends the <Form /> and <Form.Field /> components more to enable advanced use cases that we require for implementing new end device onboarding (#4847)

Changes

• Add support for "composite fields". Composite fields can map to multiple form values, allowing complex modifications with the help of encoders/decoders without the need for intermediate logic (like using _ fields).
• Add a valueSetter prop to fields which allows for full control over how a field updates form values. This is useful when a field can modify values of other independent fields, e.g. to change a value that would otherwise become invalid due to value constraints. What is important here is that this allows values to change in one single update rather than consecutively like when using useEffect() to hook into form value changes.
• Allow passing a hiddenFields prop, to avoid stripping such fields when cleaning the form values during submit.
• Compose cleaned values and provide the to the submit handler
  • Cleaned values have been stripped from all values corresponding to unmounted (but not hidden) fields, as well as intermediate fields (that we mark with _)
  • To make the validation run against the cleaned values, the validateAgainstCleanedValues needs to be used. This is only done for backwards compatibility until we refactored fields to always work with cleaned values.
• Pass the old value to the encoder to allow composing new values based on the old one. This is useful when using object-values.

Testing

I tested this heavily on the device onboarding branch. The existing e2e should verify that there are no issues with backward compatibility.

Regressions

Possible since this touches on forms.

Notes for Reviewers

Composite fields are a really nice and idiomatic way to enable complex form value modifications without the need of intermediate logic:

const initialValues = { foo: 'one', bar: 'two', baz: 'three' }
const foobarEncoder = inputValue => { foo: inputValue.split(',')[0], bar: inputValue.split(',')[1] }
const foobarDecoder = ({ foo, bar }) => [foo, bar].join(',')

<Form
  initialValues={initialValues}
  …
>
  <Form.Field
    name="foo,bar"
    component={Input}
    encode={foobarEncoder}
    decode={foobarDecoder}
    …
  />
  …
</Form>

Observe the following:

• The encoders get an object containing (only) the desired fields, as defined in the name prop
• The form will automatically spread the encoded result over the existing form values
• Both error's and touched-state is automatically mapped back to the corresponding field
• This allows updating multiple form values at the same time without intermediate steps or logic

Checklist

• Scope: The referenced issue is addressed, there are no unrelated changes.
• Compatibility: The changes are backwards compatible with existing API, storage, configuration and CLI, according to the compatibility commitments in README.md for the chosen target branch.
• Documentation: Relevant documentation is added or updated.
• Changelog: Significant features, behavior changes, deprecations and fixes are added to CHANGELOG.md.
• Commits: Commit messages follow guidelines in CONTRIBUTING.md, there are no fixup commits left.

[kschiffer]

Based on all the recent work on improving and extending our form logics, I also created the following best practices:

1. Do not use transformations in the validation schema, only validation logic:

   • The validation schema should solely focus on validating the values provided and not transform them. Any required transformations should be handled separately, ensuring the schema remains straightforward and purely for validation.

2. Keep form values in backend-shape:

   • Maintain the structure of the form values as expected by the backend. This consistency helps in minimizing the need for additional processing or transformations when sending data to the backend.

3. Use encoder/decoder to marshall values between form-field and backend shape:

   • Implement encoder and decoder functions to convert form field values to the backend format (and vice versa). This separation of concerns keeps form components clean and ensures that form data is in the correct shape when interacting with the backend.

4. When a field indirectly changes another form value, do not use onChange() or useEffect() but use valueSetter() to make all required changes in a single update:

   • Avoid using onChange or useEffect for interdependent form fields. Instead, use a valueSetter function to handle all necessary changes in one update. This approach ensures that form state updates are atomic and reduces the risk of inconsistent states.

5. Use composite fields name="foo,bar" to let form fields control multiple form values at once and do not use intermediate (name="_foobar") form values to do so:

   • Composite fields allow a single form field to manage multiple form values directly. This practice avoids using intermediate form values, simplifying the form state management and improving readability.

6. Use the cleaned form values (that strip all underscore _foo values) whenever possible onSubmit(values, formikBag, cleanedValues):

   • Always prefer using cleaned form values, which exclude any temporary or pseudo values (those starting with _) and values that correspond to hidden form fields. This ensures that only relevant data is processed and sent to the backend.

7. Use the useFormContext() to get the current form values if you want to render conditionally based on them and do not duplicate form values using change handlers and component state:

   • Leverage useFormContext to access the current form values for conditional rendering. Avoid duplicating form values in component state, as this can lead to inconsistencies and redundant logic.

8. When using setValues, always use the factory function setValues(values => {…}) instead of passing in values from the context to avoid race conditions:

   • Always use the factory function syntax (setValues(values => { ... })) to update form values. This approach ensures that you work with the most recent state and avoid potential race conditions that can arise from stale state references.

9. Do not pass stack configuration as props, always select them directly:

   • Avoid passing configuration settings as props. Instead, access these settings directly within the components where they are needed. This practice reduces the complexity of prop drilling and makes the components more self-contained.

10. Use hiddenFields-prop to exclude form values from being stripped from the cleaned values:

    • Utilize the hiddenFields prop to specify which form values should be retained even if their respective input fields are hidden. By default the form will remove these values via cleanedValues.