[crunz-ailab]

Problem and cause

Table.to_markdown(clean=True) escapes markup generated by the exporter itself.
This makes real line breaks display as literal <br> and ordinary hyphens or
minus signs display as literal -. The issue occurs in both headers and body cells.

Headers — Table.to_markdown(), src/table.py:1831–1833:

name = name.replace("\n", "<br>")
if clean:
    name = html.escape(name.replace("-", "&#45;"))

A real newline is first converted to <br>. html.escape() then turns that
generated tag into <br>, so the renderer displays the tag as text instead
of inserting a line break.

The same expression first converts - to -, then escapes the entity's
ampersand, producing -. Rendering therefore shows the literal -
rather than a hyphen, including in negative numbers.

Body cells — extract_cells(), line 380, followed by Table.to_markdown(), lines 1851–1852:

extract_cells(..., markdown=True) has already inserted <br> between text lines.
The later whole-cell call html.escape(cell.replace("-", "-")) causes the
same two problems. Escaping source text and escaping generated markup have been
combined into one operation.

Fix

• Escape header text before converting newlines to <br>.
• Pass clean into extract_cells() and escape source span text before combining
  it with generated line breaks and font formatting.
• Keep hyphens literal instead of introducing an intermediate HTML entity.

Source strings such as A<br>B and A-B still display literally. Font-derived
formatting and the default clean=False output are preserved.

Reproduction

The attached markdown_export_comparison.pdf uses the same row IDs across three pages:

1. Input strings: verified extracted values in code notation; \n represents
   an actual newline.
2. Expected display: a reference for how those values should look, not output
   from the patched library.
3. Actual output: the unmodified library's clean=True result rendered as Markdown.

Compare ID 01 for newlines, ID 02 for hyphens and negative numbers, and the
multiline header. ID 14 is the literal <br> control that must remain text.
Rows 03–10 and the additional header-pipe example illustrate separate Markdown
issues; this patch addresses the newline and hyphen defects.

reproduce_markdown.py generates the input PDF, verifies extracted values and
prints the exact Markdown returned by the installed PyMuPDF version.
Run from the directory containing the script:

python reproduce_markdown.py --output-dir reproduction_output

Tests and environment

• New regression tests: 13 failed / 16 passed before the fix; all 29 pass
  after it. They cover headers and body cells, newlines, hyphens/negative numbers,
  literal tags/entities, font styles, internal/external headers, empty-header names,
  merged cells, both fill_empty settings and unchanged clean=False output.
• Existing tests/test_tables.py: all 46 pass before and after.
• Additional checks: 57 function-level symbol cases checked with markdown-it-py
  and Mistune, plus six real tables from three upstream fixtures and three local
  manuals. The six-table checks restore line breaks without changing table
  geometry, extracted text or default Markdown.

Environment: Python 3.12.2, pytest 7.4.4, saved main Python sources with released
PyMuPDF 2.0 / MuPDF 1.28.5 compiled components. The branch base 16c664f has the
same Git tree as the tested 773a0d9 snapshot. The PDF comparison uses
markdown-it-py 2.2.0 and Chromium; the regression tests themselves require no
Markdown renderer.

Additional tests/test_markdown_support.py checks have the same 2 failures and
1 setup error before and after: the local checkout lacks nur-ruhig.jpg and
test_markdown_bad_unicode.md, and the missing-image warning affects the following
test's setup.

ATTACHMENT_NOTES.txt

markdown_export_comparison.pdf
reproduce_markdown.py

[veget-able]

Thanks for the fix. Escaping source text before adding generated markup addresses the line-break and hyphen issues directly.

In my 168-page sample, default Markdown was unchanged across all 421 table outputs from the layout and non-layout runs. With clean=True, generated line breaks and ordinary hyphens are now preserved. Cell extraction and header metadata were unchanged as well.

Looks good to me. One small documentation suggestion: could the clean docstring say that it HTML-escapes source text while preserving generated line breaks and formatting? That would describe the behaviour more accurately than saying Markdown syntax is removed.