DocMaster

A Hierarchical Structure-Aware System for Document Analysis

Paper · Project website · Web demo

DocMaster is a research system for filtering and analyzing collections of complex PDF documents. Instead of flattening each document into independent text chunks, DocMaster preserves structural elements—such as sections, tables, figures, and equations—in a hierarchical document tree. It augments this representation with semantic clusters and hyperedges to support natural-language document filtering and retrieval-augmented question answering.

This repository contains the reference implementation and interactive demo accompanying the paper:

Ziqi Chen, Yingli Zhou, Fangyuan Zhang, Quanqing Xu, Chuanhui Yang, and Yixiang Fang. “DocMaster: A Hierarchical Structure-Aware System for Document Analysis.” arXiv:2607.08539, 2026.

Key capabilities

• Parse PDFs into layout-aware hierarchical document trees with MinerU.
• Construct complementary document-tree, semantic-cluster, and hyperedge indexes.
• Filter document collections using natural-language conditions.
• Compare retrieval strategies and their LLM token usage.
• Explore document structures and semantic indexes interactively.
• Ask follow-up questions over all documents or the filtered subset.

System overview

DocMaster follows two main stages:

1. Index construction: PDF parsing preserves document structure; DocMaster then builds hierarchical, semantic, hyperedge, and FAISS-based retrieval indexes.
2. Document analysis: a natural-language condition retrieves evidence from the indexes and determines which documents match. The resulting subset can be analyzed further through structure-aware RAG.

Repository structure

.
├── app.py                    # Flask API and local demo server
├── frontend/                 # Interactive web interface
├── requirements-web.txt      # Web-server dependencies
└── src/
    ├── Document/             # Document nodes, trees, and PDF extraction
    ├── RAG/                  # Hierarchical retrieval and RAG
    ├── builder/              # Tree, semantic index, and keyword builders
    ├── prompts/              # LLM prompt templates
    ├── provider/             # LLM and embedding providers
    ├── config.py             # Typed configuration schema
    ├── default.yaml          # Example runtime configuration
    ├── filter.py             # LLM-based filter operator
    └── requirements.txt      # Core Python dependencies

Installation

The reference environment uses Python 3.11. A GPU is recommended for document parsing and embedding, but the included dependencies use CPU FAISS by default.

git clone https://github.com/ChenziqiAdam/DocMaster.git
cd DocMaster

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r src/requirements.txt -r requirements-web.txt

Configuration

Before running DocMaster, edit src/default.yaml. At minimum, configure:

• llm: an OpenAI-compatible or Ollama language-model endpoint.
• vlm: the vision-language model used during document extraction.
• mineru: the MinerU backend, method, language, and optional server URL.
• filter.condition: the default natural-language filtering condition.

The checked-in configuration contains placeholder credentials and example service endpoints. Do not commit API keys. Backend services must be running before documents are uploaded. See the MinerU documentation for supported parsing backends and deployment instructions.

Run the local demo

Start the Flask application from the repository root:

python app.py

Then open http://localhost:5000. The interface supports the following workflow:

1. Upload one or more PDF documents (up to 50 MB each).
2. Inspect the extracted document tree and semantic index.
3. Enter a natural-language document filter.
4. Compare document-tree, semantic-cluster, hyperedge, and combined results.
5. Ask follow-up questions over the full collection or matched documents.

Generated uploads and indexes are stored under uploads/ and are excluded from version control.

Reproducibility notes

• Index construction invokes external model services and can be computationally intensive.
• Results can vary with the selected LLM/VLM, model versions, and decoding settings.
• Record the effective src/default.yaml, model identifiers, and random seeds when reporting experiments.
• The current repository is a research prototype; review configuration and data-handling requirements before deployment.

Citation

If you use DocMaster in your research, please cite:

@article{chen2026docmaster,
  title   = {DocMaster: A Hierarchical Structure-Aware System for Document Analysis},
  author  = {Chen, Ziqi and Zhou, Yingli and Zhang, Fangyuan and Xu, Quanqing and Yang, Chuanhui and Fang, Yixiang},
  journal = {arXiv preprint arXiv:2607.08539},
  year    = {2026},
  doi     = {10.48550/arXiv.2607.08539}
}

License

No open-source license is currently included. Please contact the authors regarding use beyond research evaluation until a license is published.