Add optional sanitization of illegal XLSX characters on export (fixes #370)#653
Open
apoorvdarshan wants to merge 1 commit into
Open
Add optional sanitization of illegal XLSX characters on export (fixes #370)#653apoorvdarshan wants to merge 1 commit into
apoorvdarshan wants to merge 1 commit into
Conversation
Exporting a Dataset whose string cells contain control characters that are illegal in XLSX (matched by openpyxl's ILLEGAL_CHARACTERS_RE, e.g. '\x1f') raised openpyxl.utils.exceptions.IllegalCharacterError with no way to produce a file. Add a ``sanitize_illegal_chars`` parameter to ``export_set()`` and ``export_book()``. When set to a string, illegal characters are replaced with it (``""`` strips them); when left as ``None``/``False`` (the default), the exception still bubbles up, preserving existing behavior. The offending characters are only substituted when openpyxl raises, so the common (clean) path is unaffected. Fixes jazzband#370.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Exporting a
Dataset(orDatabook) whose string cells contain control characters that are illegal in XLSX crashed withopenpyxl.utils.exceptions.IllegalCharacterErrorand gave callers no way to produce a file (issue #370):These are the characters matched by openpyxl's
ILLEGAL_CHARACTERS_RE([\000-\010]|[\013-\014]|[\016-\037]).Fix
Added an optional
sanitize_illegal_charsparameter toexport_set()andexport_book()insrc/tablib/formats/_xlsx.py, following the design discussed by the maintainers on the issue (a string replacement char, withNone/Falsemeaning "raise", and substituting only when openpyxl actually raises rather than regex-scanning every cell):None, orFalse): behavior is unchanged — theIllegalCharacterErrorstill bubbles up, sotest_xlsx_wrong_charand existing callers are unaffected.ILLEGAL_CHARACTERS_RE.sub(...). Use""to strip them (an empty string is treated as a valid replacement, not as "raise").The substitution happens inside the existing
tryaroundcell.value = col, so the common clean path pays no extra cost.Tests
Added to
tests/test_tablib.py:test_xlsx_export_illegal_char_raises_by_default— the exact issue repro; confirms the default still raisesIllegalCharacterError.test_xlsx_export_set_sanitize_illegal_chars— strip ("") and replace ("?") paths forexport_set.test_xlsx_export_book_sanitize_illegal_chars— the replacement path forexport_book.The two sanitize tests fail on pristine
master(the parameter did not exist) and pass with the fix. All existing xlsx tests, includingtest_xlsx_wrong_char, remain green.ruffpasses on the changed files. Docs (docs/formats.rst) andAUTHORSupdated.Disclosure: prepared with AI assistance; reviewed and verified locally.