[Bug]: Certain PDF crashes RAG pipeline

[rjakomin]

Steps to reproduce

Hi,
whenever I try to copy the attached PDF file to my data folder monitored by the private RAG pipeline, it crashes the engine (the whole app docker container) without any error message. It happens every time for this pdf document: https://www.dancilla.com/PDF/Dancilla_alle_Volkstaenze.pdf

Relevant log output

2025-02-25 14:59:40 pathway_engine.connectors.monitoring INFO FileSystem(data): 1 entries (3530 minibatch(es)) have been sent to the engine
2025-02-25 15:00:32 root INFO {"_type": "request_payload", "session_id": "uuid-29d4de7b-6cd3-4f92-ab6e-5111029c3157", "payload": {}}
2025-02-25 15:00:37 root INFO {"_type": "request_payload", "session_id": "uuid-65828918-7085-4c99-8385-7a9c93320895", "payload": {}}
2025-02-25 15:00:44 pathway_engine.connectors.monitoring INFO FileSystem(data): 0 entries (1 minibatch(es)) have been sent to the engine
2025-02-25 15:00:44 pathway_engine.connectors.monitoring INFO PythonReader: 2 entries (87119 minibatch(es)) have been sent to the engine
2025-02-25 15:01:02 pathway_engine.connectors.monitoring INFO PythonReader: 0 entries (5 minibatch(es)) have been sent to the engine

What did you expect to happen?

Processing of the newly copied PDF file and including its content into the vector database used by the private RAG.

Version

current

Docker Versions (if used)

27.4.0, build bde2b89 (running on Windows 11)

OS

Windows 11

[dxtrous]

Thank you for the report @rjakomin. While we investigate this, could you share any relevant statistics (like docker stats memory, CPU usage profile directly before the crash) which could explain the cause of the crash?

[XGendre]

Hi @rjakomin , thanks for your report. I did multiple tests in various environments including a Docker container in Windows 11. No problem occurs with the PDF file you gave. The file is read correctly and there is no crash.

Could you provide additional details about your environment and Docker statistics @dxtrous mentioned?