[Bug]: Parsing some files will result in an error chunking: cannot convert float NaN to integer

[chanjiashuo]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

RAGFlow workspace code commit ID

7eb25e0

RAGFlow image version

nightly-slim

Other environment information

Actual behavior

Page: [ERROR]Internal server error while chunking: cannot convert float NaN to integer
[ERROR][Exception]: cannot convert float NaN to integer

Expected behavior

No response

Steps to reproduce

Parse files in the dataset

Additional information

No response

[dosubot]

The error "cannot convert float NaN to integer" during chunking happens because the chunking logic in RAGFlow doesn't check for NaN values before converting floats to integers. This usually means that some earlier calculation (like chunk size, step, or token count) produced a NaN, often due to invalid or empty input data. There are no recent code changes or fixes for this in the current nightly-slim image or around your commit.

To diagnose and work around this, try inspecting the intermediate parse output or chunk preview to see if the parser is producing unexpected results. If you're working with CSV or table data, check for problematic quoted fields or embedded newlines. Reducing the chunk size or converting your files to JSONL (one record per line) can also help isolate or avoid the issue. More details and step-by-step suggestions are available in this related issue.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[chanjiashuo]

@dosu The file types I reported errors in are all pdf

[dosubot]

The "cannot convert float NaN to integer" error during PDF chunking in RAGFlow is caused by NaN values propagating from earlier parsing steps—often due to corrupt, empty, or malformed data in the PDF. The chunking logic does not check for NaN before converting floats to integers, so any invalid coordinate, size, or token calculation can trigger this crash. This is a known limitation in the current codebase, and there are no recent fixes for it in the nightly-slim image or around your commit source.

To troubleshoot: try isolating problematic PDFs by parsing them one at a time, inspect intermediate parse outputs for missing or obviously broken text/coordinates, and consider preprocessing your PDFs (e.g., re-saving with a different tool, removing blank pages, or flattening annotations). If you have control over chunking config, you can experiment with different chunking strategies (token count, delimiters, section headers) source. For persistent issues, using CPU-only mode (not GPU) is recommended for PDF parsing source.

If you need to avoid the error for now, try converting the PDF to a clean text format or JSONL and parse that instead, or reduce chunk size to see if the error is triggered by specific large or empty sections.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[KevinHuSh]

Could you share the file with us? kai.hu@infiniflow.org

[chanjiashuo]

@KevinHuSh Hello, the file has been sent to the mail. This problem will occur when the file parser is selected as deepdoc

[KevinHuSh]

I don't think it's the file that cause problems. The problem might be caused by embedding inference.

[chanjiashuo]

@KevinHuSh This problem should not be a problem with the embedding model, because I can parse it successfully by changing the parser from deepdoc to naive. I parsed about 100 documents, and 9 of them had this error. The dataset is parsed by deepdoc by default.