771: restore pre-py3.13 xml attribute escaping behaviour for \r\n\t#772
Conversation
- in py3.13, minidom started escaping \r\n\t in attributes in order to comply with the XML spec, which pyxform adopted for all python versions in an earlier commit. But it was found that this change caused a HTTP 500 error for Enketo (not Collect/Webforms). If a user had choice filter in a multi-language form, and the filter expression used `select()`, then Enketo could not display the form but displayed an error saying "Cannot read properties of undefined ( reading length)". Details in XLSForm/pyxform/XLSForm#771 - many test cases were explored such as constraint, calculate, relevant, single or multi-language forms, choice filter expressions and operators, but only the one mentioned above seemed to trigger the error. Since it seems a reasonably common trigger (using newlines to format expressions, and using `selected()` in choice filters), and it may take some time for both Enketo to fix the issue and for that release to be adopted by users, pyxform reverts to the previous behaviour.
lognaturel
left a comment
There was a problem hiding this comment.
Double checking my understanding -- when I reviewed the original PR my understanding was that by virtue of using Python 3.13, the escaping would happen automatically for pyxform. And then I thought the changes that you're undoing here were just to make other Python versions consistent. But seeing this now, it looks like this escaping was always custom for all Python versions, is that correct?
|
@ln yes the custom implementations are there to achieve output that is consistent / compliant XML / compatible with Enketo across python versions. Even if there was no Enketo issue, custom implementations would still have been needed:
Also the standard library escaping algo does repeated searches/calls like |
Closes #771
Why is this the best possible solution? Were any other approaches considered?
In py3.13, minidom started escaping
\r\n\tin attributes in order to comply with the XML spec, which pyxform adopted for all python versions in an earlier commit. But it was found that this change caused a HTTP 500 error for Enketo (not Collect/Webforms). If a user had choice filter in a multi-language form, and the filter expression usedselect(), then Enketo could not display the form but displayed an error saying "Cannot read properties of undefined ( reading length)". Details in XLSForm/pyxform/#771 - many test cases were explored such as constraint, calculate, relevant, single or multi-language forms, choice filter expressions and operators, but only the one mentioned above seemed to trigger the error. Since it seems a reasonably common trigger (using newlines to format expressions, and usingselected()in choice filters, within a multi-language form), and it may take some time for both Enketo to fix the issue and for that release to be adopted by users, pyxform reverts to the previous behaviour.What are the regression risks?
It seems like there are a few XForm generator tools out there so if they adopt python 3.13 they may hit the same issue, but that is not really a pyxform problem as such - they could follow/comment/fix the issue enketo/enketo#1408
Does this change require updates to documentation? If so, please file an issue here and include the link below.
No, it seems a bit too detailed an unlikely an issue to mention anywhere that pyxform isn't escaping
\r\n\tin attribute values. If needed, a forum post could reference this PR and it's linked issues.Before submitting this PR, please make sure you have:
testspython -m unittestand verified all tests passruff format pyxform testsandruff check pyxform teststo lint code