Getting the rough sequence of text and images on a page: How-to?

[skye0402]

I want to get the order of text and images on a page. I'm very happy with what I get from that:

    for page_num in range(len(document)):
        page = document.load_page(page_num)
        blocks = page.get_text("blocks", sort=False, flags = TEXTFLAGS_BLOCKS | TEXT_PRESERVE_IMAGES)
        images = page.get_images(full=True)
...

Problem is, that the images in the list are just like that:
(498.94989013671875, 40.149940490722656, 576.7498779296875, 78.64190673828125, '<image: Indexed(255,DeviceRGB), width: 175, height: 86, bpc: 8>', 1, 1)

And the image data has IDs (I believe it's '1978' in this case):
(1978, 1983, 175, 86, 8, 'Indexed', '', 'Im0', 'FlateDecode', 0)

At the end I want a JSON list of text/image/text/image in the sequence of the blocks but with base64 data of the images. I'm not clear how I should identify the image in the images list from the blocks list entry as there is no id?

Yes, I could use "dict" but then I get e.g. "8th October" separated into "8", "th", "October" and lose the nice layout I get from blocks.

Any idea?

[JorjMcKie]

The sequence of the blocks you get from get_text("blocks", flags=pymupdf.TEXTFLAGS_DICT) on the one hand and get_text("dict")["blocks"] on the other hand are the same.

The dictionaries of the second list have a key "number" which is the sequence number of the respective block during extraction. This equals the value of block[-2] in the first list.
So you are able to run through the first list and access the image details in the second list with that block number.

As per the image data contained in the second list, please look at this. The value of the "image" key are the binary data of the image (bytes object).
Like always, you can convert that bytes object with the standard base64 encoding modules of Python.

Or, if you want to be lazy, use "json" instead of "dict" for extraction. Then that encoding has already been done for you.
You must of course convert the JSON string output back to a Python dictionary. Then access the "image" key as a base64 string.

[skye0402]

@JorjMcKie thank you kindly for the detailed answer, I have amended my code now to this:

blocks = page.get_text("blocks", sort=False, flags=TEXTFLAGS_DICT | TEXT_PRESERVE_IMAGES)
images = page.get_images(full=True)   

As for your explanation with counting the index I'm too dumb to understand. I attach 2 screenshots:

and the images:

I understand I can just follow the sequence so first image in the blocks is the first image in images - is that what you meant?
If I can rely on that then I will do this way (unless there's a better approach, currently I compare the images sizes 😃).

Also I'd like to thank you for the answer you provided already! Not taken for granted!

[JorjMcKie]

Yes, all variants of page.get_text(variant, ...) use the same underlying PyMuPDF object TextPage.
A textpage is the object from which all data are extracted and on which also page.search_for() operates. TextPage creation is the only code that actually accesses the PDF file / page.

However, it is important to note that from the same page different versions of a TextPage can be created (e.g. with or without images, or with different "clips" = sub-rectangles).

The get_text() variants under the hood create a temporary TextPage if not provided (via the textpage parameter). And that temporary TextPage is optimized per variant: variants "text", "blocks" and "words" each create a TextPage that ignores images. "dict", "rawdict" and others include images.

The content of a TextPage is controlled by two parameter: clip and flags.

There are prepared flags combinations per variant: TEXTFLAGS_TEXT, TEXTFLAGS_DICT, TEXTFLAGS_WORDS, etc.
So, in order to make sure that the same number of blocks exist across different variants, we need to either prepare a TextPage separately and provide it to all get_text() calls, or create the same TextPage each time.

So in your special case, the best and fastest approach would be this:

tp = page.get_textpage(flags=pymupdf.TEXTFLAGS_DICT)

# then do these get_text calls:
std_blocks = page.get_text("blocks", textpage=tp)
dict_blocks = page.get_text("dict", textpage=tp)["blocks"]

Both above extractions have identical number and sequence of blocks.

[skye0402]

There seems to be a difference though using your proposed approach:

versus

I think it's the first entry in std_blocks that is not in dict_blocks (maybe the background JPEG).