Skip to content

Add optional sanitization of illegal XLSX characters on export (fixes #370)#653

Open
apoorvdarshan wants to merge 1 commit into
jazzband:masterfrom
apoorvdarshan:fix-xlsx-illegal-char-370
Open

Add optional sanitization of illegal XLSX characters on export (fixes #370)#653
apoorvdarshan wants to merge 1 commit into
jazzband:masterfrom
apoorvdarshan:fix-xlsx-illegal-char-370

Conversation

@apoorvdarshan

Copy link
Copy Markdown

Summary

Exporting a Dataset (or Databook) whose string cells contain control characters that are illegal in XLSX crashed with openpyxl.utils.exceptions.IllegalCharacterError and gave callers no way to produce a file (issue #370):

from tablib import Dataset
data = Dataset()
data.append(('\x1f',))
data.export('xlsx')  # IllegalCharacterError

These are the characters matched by openpyxl's ILLEGAL_CHARACTERS_RE ([\000-\010]|[\013-\014]|[\016-\037]).

Fix

Added an optional sanitize_illegal_chars parameter to export_set() and export_book() in src/tablib/formats/_xlsx.py, following the design discussed by the maintainers on the issue (a string replacement char, with None/False meaning "raise", and substituting only when openpyxl actually raises rather than regex-scanning every cell):

  • Default (None, or False): behavior is unchanged — the IllegalCharacterError still bubbles up, so test_xlsx_wrong_char and existing callers are unaffected.
  • A string value: illegal characters in string cell values are replaced with it via ILLEGAL_CHARACTERS_RE.sub(...). Use "" to strip them (an empty string is treated as a valid replacement, not as "raise").

The substitution happens inside the existing try around cell.value = col, so the common clean path pays no extra cost.

data.export('xlsx', sanitize_illegal_chars='')   # strip
data.export('xlsx', sanitize_illegal_chars='?')  # replace

Tests

Added to tests/test_tablib.py:

  • test_xlsx_export_illegal_char_raises_by_default — the exact issue repro; confirms the default still raises IllegalCharacterError.
  • test_xlsx_export_set_sanitize_illegal_chars — strip ("") and replace ("?") paths for export_set.
  • test_xlsx_export_book_sanitize_illegal_chars — the replacement path for export_book.

The two sanitize tests fail on pristine master (the parameter did not exist) and pass with the fix. All existing xlsx tests, including test_xlsx_wrong_char, remain green. ruff passes on the changed files. Docs (docs/formats.rst) and AUTHORS updated.

Disclosure: prepared with AI assistance; reviewed and verified locally.

Exporting a Dataset whose string cells contain control characters that
are illegal in XLSX (matched by openpyxl's ILLEGAL_CHARACTERS_RE, e.g.
'\x1f') raised openpyxl.utils.exceptions.IllegalCharacterError with no
way to produce a file.

Add a ``sanitize_illegal_chars`` parameter to ``export_set()`` and
``export_book()``. When set to a string, illegal characters are replaced
with it (``""`` strips them); when left as ``None``/``False`` (the
default), the exception still bubbles up, preserving existing behavior.
The offending characters are only substituted when openpyxl raises, so
the common (clean) path is unaffected.

Fixes jazzband#370.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant