Repository navigation
fix: replace inaccessible PDF in RAG example - #1732
Open
jeeth-kataria wants to merge 2 commits into
Open
jeeth-kataria wants to merge 2 commits into
jeeth-kataria wants to merge 2 commits into
Conversation
Use the dated W3C WCAG 1.0 checklist for document-grounded QA. Document prerequisites and add an explicit structural e2e example check. Fixes generative-computing#373 Assisted-by: Codex Signed-off-by: Jeeth <jeethkataria9798@icloud.com>
rafvasq
requested changes
Oct 16, 2026
Contributor
There was a problem hiding this comment.
Thanks @jeeth-kataria! The script and test change look good to me. For the example's README, I think that the documentation can be kept short and consistent with how other examples are described you can minimize the scope of this PR to simply updating the document used and example using it because the README will need to be re-organized (outside of this PR).
Comment on lines
+18
to
+65
| Document-grounded question answering using W3C's | ||
| [WCAG 1.0 checklist PDF](https://www.w3.org/TR/1999/WAI-WEBCONTENT-19990505/full-checklist.pdf). | ||
| The question asks what the three checkpoint priority levels mean; the answer is | ||
| in the document's **Priorities** section on page 2. In brief, developers must | ||
| satisfy Priority 1 checkpoints, should satisfy Priority 2 checkpoints, and may | ||
| address Priority 3 checkpoints. Generated wording can vary. | ||
|
|
||
| This dated, seven-page publication is historical sample material, not current | ||
| accessibility guidance. It replaces a USDA PDF that returned HTTP 403 for users | ||
| outside the US. The PDF remains hosted by W3C under its | ||
| [document license](https://www.w3.org/Consortium/Legal/copyright-documents-19990405); | ||
| no copy is bundled with this example. | ||
|
|
||
| **Prerequisites:** Install the Docling extra, have internet access to download | ||
| the PDF and Docling's conversion models, and run Ollama with `granite4.2:3b` | ||
| (the current `start_session()` default). FAISS and sentence-transformers are | ||
| not needed for this example. OCR is disabled because the PDF contains text; | ||
| Docling may still download layout and table models on the first conversion. | ||
|
|
||
| From the repository root: | ||
|
|
||
| ```bash | ||
| uv sync --frozen --extra docling | ||
| ollama serve # run in a separate terminal if Ollama is not already running | ||
| ollama pull granite4.2:3b | ||
| uv run python docs/examples/rag/mellea_pdf.py | ||
| ``` | ||
|
|
||
| From another working directory, use absolute paths to the checkout and script: | ||
|
|
||
| ```bash | ||
| uv run --project /path/to/mellea python /path/to/mellea/docs/examples/rag/mellea_pdf.py | ||
| ``` | ||
|
|
||
| The document URL is independent of the working directory. PDF conversion happens | ||
| before the session is created, so document-loading failures can be distinguished | ||
| from model-inference failures. The `e2e, ollama, slow` markers opt the example | ||
| into explicit slow-test runs. Its assertions check only that the result is a | ||
| non-empty string, not the factual content of the answer. The `slow` marker | ||
| accounts for cold conversion-model downloads and inference; ordinary test runs | ||
| exclude it. `CICD=1` does not skip this structural test, but current PR CI does | ||
| not collect Python examples. | ||
|
|
||
| To run it through pytest, including the slow marker: | ||
|
|
||
| ```bash | ||
| uv run pytest docs/examples/rag/mellea_pdf.py -m "e2e and ollama" --no-cov -v -s | ||
| ``` |
Contributor
There was a problem hiding this comment.
Extra context not needed at the moment. I think this can be kept to one-line for now as it was before.
Contributor
There was a problem hiding this comment.
+1 - rest of fix looks good, but this is unnecessarily verbose and historic
planetf1
reviewed
Oct 20, 2026
planetf1
left a comment
planetf1
left a comment
Contributor
There was a problem hiding this comment.
all good other than the need to trim the long comment
Contributor
|
Hi @jeeth-kataria! Once the small requested change is in, we'll be good to merge this. Let me know if you'll have time soon, or if you'd like me to make the edit to help get it merged. |
Retain upstream documentation structure and setup instructions. Resolve the README description to one line as requested in review. Assisted-by: Codex Signed-off-by: Jeeth <jeethkataria9798@icloud.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Issue
Fixes #373
Description
Replace the USDA PDF (HTTP 403 from our Indian network) with W3C's dated WCAG 1.0 checklist. Ask about its three priority definitions, update the grounding label, and disable OCR for this text PDF. Keep the README description to one line in upstream's table, preserving upstream's setup docstring in the script. The document remains externally hosted; no binary or dependency is added.
Add structural assertions and
e2e, ollama, slowmarkers. Also correct the default-model pull instruction intest/README.md, as suggested by the maintainer. No library, adapter or workflow code changes.The maintainer waived US verification and requested this PR in the issue discussion. No US run was performed; worldwide accessibility is not claimed.
Testing
Actual local validation:
mellea.start_session()'s defaultgranite4.2:3b, without model overrides. Recorded prompt includes the complete document underwcag_checklist; response metadata confirms Granite. Root and absolute-path execution from another directory succeeded. Earlier directory checks substituted inference, and earlier Llama runs used an override; neither is the default-model evidence.CICD=1 HF_HUB_DISABLE_XET=1 uv run --no-sync pytest docs/examples/rag/mellea_pdf.py -m 'e2e and ollama' --no-cov -v -s: 1 passed in 44.22s. Latest answer correctly states all three definitions, one sentence each. Semantic review is manual; an earlier run added less precise qualifiers, so structural success does not guarantee answer quality.uv run --no-sync pytest test/test_example_collection.py --no-cov -q: 20 passed in 14.32s after conflict resolution.uv run --no-sync pre-commit run --files docs/examples/rag/README.md docs/examples/rag/mellea_pdf.py test/README.md: applicable hooks passed, including full mypy, after conflict resolution. Original submission's--all-filesrun also passed; not repeated this round. Initial environment repair installed locked dev extras and compatible local-only stubs, restoring locked NumPy/Requests. No dependency files changed in this PR.git diff --cached --check: passed before commit.Original submission's broader suite attempt (
CICD=1 HF_HUB_DISABLE_XET=1 uv run --no-sync pytest test --no-cov --maxfail=1 -q) stopped on the 900s uncertainty-adapter test timeout: 646 passed, 62 skipped, 136 deselected, 1 xpassed, 1 error. Not rerun for this review update; no complete-suite pass or updated-head CI result is claimed.HF_HUB_DISABLE_XET=1was used for verification after an hf-xet conversion-weight download stalled; it is not in the example.Marker rationale
The example checks only string type and non-empty output, not generated facts. Cold conversion-weight/model downloads justify
slow; explicit-m 'e2e and ollama'overrides the default slow exclusion.CICD=1skips qualitative items, not this structural check.Current PR CI installs Docling and pulls Granite but does not collect Python examples. Notebook CI collects notebooks only; nightly defaults exclude this slow example. This enables explicit execution, not new automated PR-CI coverage.
Attribution
Codex assisted with investigation, code and validation. Commit uses the human author's existing Git identity, DCO sign-off and
Assisted-by: Codex.Adding a new component, requirement, sampling strategy, or tool?
If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.
NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.