Unterminated object when using PdfWriter with incremental=True

[m32]

Environment

Linux-6.1.0-31-amd64-x86_64-with-glibc2.36
pypdf==5.3.0, crypt_provider=('cryptography', '44.0.0'), PIL=11.1.0

Code + PDF

This is a minimal, complete example that shows the issue:

import fpdf
doc = fpdf.FPDF()
doc.set_compression(0)
font = 'helvetica'

for i in range(2):
    doc.add_page()
    doc.set_font(font, '', 13.0)
    link = None
    doc.cell(w=75.0, h=22.0, align='C', text='Hello, world page=%d.' % i, border=0, new_x=fpdf.XPos.LEFT, new_y=fpdf.YPos.NEXT, link=link)
doc.output('pdf.pdf')

import io
from pypdf import PdfReader, PdfWriter

stream = io.BytesIO()
writer = PdfWriter('pdf.pdf', incremental=True)
writer.add_blank_page(72, 72)
writer.write(stream)
with open('pdf-out.pdf', 'wb') as fp:
    fp.write(stream.getvalue())

Last object in generated pdf is XRef table but it has no endobj tag:
12 0 obj<<
/Type /XRef
...
/Length 18

stream
....
endstream
endobj <- this element is missing
startxref
1742
%%EOF

[stefan6419846]

Thanks for the report. Feel free to submit a corresponding PR.

[m32]

I am the author of the endesive package (cryptographic signature of pdf files), hence my interest in incremental saving to a pdf file.
When I corrected the save, acrobat reader still does not want to notice the signature in the document, and when trying to save it to a new file, it even reports internal error 26.
Normally, a pdf file trailer is not an object with the XRef type and when I converted it to something I consider normal, only then did the functions with signature support in pdf documents appear and saving to a new pdf file works correctly.

I do not know how the current trailer with incremental saving compares to the pdf standard and who is making the mistake and to whom I can report my problem.

What is your opinion on this subject?
Do you allow the old form of saving the trailer as a PR?

[stefan6419846]

I have to admit that I have never used incremental saving or really dug into signing PDF files, as there is no real use case from my side.

Apart from this: As long as your changes do not (knowingly) break existing workflows and are explained (and possibly backed by the PDF specification) well enough, I do not see an issue with merging a corresponding PR. If this relates to a past change in pypdf, I appreciate pointing to it as well to evaluate why this might have been changed.

I do not know how the current trailer with incremental saving compares to the pdf standard and who is making the mistake and to whom I can report my problem.

At least the PDF 1.7 standard is publicly available and the PDF 2.0 standard can be downloaded for free after providing your personal data. Unfortunately, not all aspects are defined well enough, thus some decisions or implementation details might not match exactly (and maybe cannot match exactly). I would assume that Adobe as the original creator of the standard are doing it as intended originally, but the lack of error debugging there usually makes it hard to identify the actual culprit.

[stefan6419846]

What is the current state of this issue from your side after you closed #3119?

[m32]

I have a problem writing a test for this report, first of all. Secondly, I would have to read the pdf documentation because I have too little knowledge about incremental recording, and this whole procedure does not suit me.
Generally, I need more time.