Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .agents/skills/quantmind-dev/references/develop-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,13 @@ apply throughout.
- Pydantic models, `frozen=True`, `extra="forbid"`.
- Every `BaseKnowledge` subclass **must** require `as_of: datetime`
(financial time-sensitivity is mandatory) and a typed `source: SourceRef`
(no bare strings), and **must** override `embedding_text()`.
(no bare strings).
- Canonical models do not select retrieval text or store vectors. Put
rebuildable text projections in `quantmind.library`.
- Pick one shape: `FlattenKnowledge` (atomic card), `TreeKnowledge`
(hierarchical artifact), or `GraphKnowledge` (placeholder). Whole-document
objects are `TreeKnowledge` even when a flatten card exists alongside
(e.g. `Paper` vs `PaperKnowledgeCard`).
(hierarchical artifact), or `GraphKnowledge` (placeholder) for conventional
knowledge. Source-first paper revisions and independently versioned paper
artifacts use their dedicated frozen models instead of `BaseKnowledge`.

### `quantmind/configs/` — operation cfg + typed inputs

Expand Down
10 changes: 6 additions & 4 deletions .claude/skills/quantmind-dev/references/develop-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,13 @@ apply throughout.
- Pydantic models, `frozen=True`, `extra="forbid"`.
- Every `BaseKnowledge` subclass **must** require `as_of: datetime`
(financial time-sensitivity is mandatory) and a typed `source: SourceRef`
(no bare strings), and **must** override `embedding_text()`.
(no bare strings).
- Canonical models do not select retrieval text or store vectors. Put
rebuildable text projections in `quantmind.library`.
- Pick one shape: `FlattenKnowledge` (atomic card), `TreeKnowledge`
(hierarchical artifact), or `GraphKnowledge` (placeholder). Whole-document
objects are `TreeKnowledge` even when a flatten card exists alongside
(e.g. `Paper` vs `PaperKnowledgeCard`).
(hierarchical artifact), or `GraphKnowledge` (placeholder) for conventional
knowledge. Source-first paper revisions and independently versioned paper
artifacts use their dedicated frozen models instead of `BaseKnowledge`.

### `quantmind/configs/` — operation cfg + typed inputs

Expand Down
34 changes: 29 additions & 5 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ on:
- 'quantmind/preprocess/fetch/arxiv.py'
- 'quantmind/preprocess/format/pdf.py'
- 'quantmind/rag/**'
- 'quantmind/configs/paper.py'
- 'quantmind/flows/_paper_summary.py'
- 'quantmind/flows/paper.py'
- 'quantmind/knowledge/paper.py'
- 'quantmind/library/**'
- 'scripts/verify_pdf_rag_e2e.py'
- 'pyproject.toml'
schedule:
Expand All @@ -38,7 +43,7 @@ jobs:
runs-on: ubuntu-latest
outputs:
news: ${{ steps.filter.outputs.news }}
pdf_rag: ${{ steps.filter.outputs.pdf_rag }}
paper_flow: ${{ steps.filter.outputs.paper_flow }}

steps:
- name: Checkout
Expand All @@ -60,9 +65,14 @@ jobs:
- 'quantmind/preprocess/format/html.py'
- 'quantmind/preprocess/fetch/**'
- 'pyproject.toml'
pdf_rag:
paper_flow:
- '.github/workflows/e2e.yml'
- 'scripts/verify_pdf_rag_e2e.py'
- 'quantmind/configs/paper.py'
- 'quantmind/flows/_paper_summary.py'
- 'quantmind/flows/paper.py'
- 'quantmind/knowledge/paper.py'
- 'quantmind/library/**'
- 'quantmind/preprocess/fetch/arxiv.py'
- 'quantmind/preprocess/format/pdf.py'
- 'quantmind/rag/**'
Expand Down Expand Up @@ -98,11 +108,13 @@ jobs:
- name: Run live news E2E
run: .venv/bin/python scripts/verify_news_e2e.py

pdf-rag:
paper-flow:
needs: changes
if: github.event_name != 'pull_request' || needs.changes.outputs.pdf_rag == 'true'
if: github.event_name != 'pull_request' || needs.changes.outputs.paper_flow == 'true'
runs-on: ubuntu-latest
timeout-minutes: 10
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

steps:
- name: Checkout
Expand All @@ -125,5 +137,17 @@ jobs:
- name: Install project runtime dependencies
run: uv pip install --python .venv/bin/python -e .

- name: Run live PDF RAG E2E
- name: Check OpenAI credential
id: openai
shell: bash
run: |
if [[ -n "${OPENAI_API_KEY}" ]]; then
echo "available=true" >> "$GITHUB_OUTPUT"
else
echo "available=false" >> "$GITHUB_OUTPUT"
echo "::notice::Skipping Paper Flow V1 E2E because OPENAI_API_KEY is not configured."
fi

- name: Run live Paper Flow V1 E2E
if: steps.openai.outputs.available == 'true'
run: .venv/bin/python scripts/verify_pdf_rag_e2e.py
39 changes: 16 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ We use [uv](https://github.com/astral-sh/uv) for fast and reliable Python packag

Component-specific guides and architecture notes live under [`docs/`](docs/).

#### Run a single paper through `paper_flow`
#### Build an exact paper source, chunks, and cited summary

```python
import asyncio
Expand All @@ -167,11 +167,12 @@ from quantmind.flows import paper_flow


async def main() -> None:
paper = await paper_flow(
ArxivIdentifier(id="2401.12345"),
result = await paper_flow(
ArxivIdentifier(id="1706.03762v7"),
cfg=PaperFlowCfg(model="gpt-4o-mini"),
)
print(paper.model_dump_json(indent=2))
print(result.global_summary.summary)
print(result.source_revision.id, result.chunk_set.id)


asyncio.run(main())
Expand Down Expand Up @@ -219,17 +220,17 @@ async def main() -> None:
"Pull arXiv 2401.12345 about cross-sectional momentum; use gpt-4o-mini.",
target_flow=paper_flow,
)
paper = await paper_flow(inp, cfg=cfg)
print(paper.model_dump_json(indent=2))
result = await paper_flow(inp, cfg=cfg)
print(result.global_summary.summary)


asyncio.run(main())
```

> **Note**: QuantMind is mid-migration to OpenAI Agents SDK
> (see [#71](https://github.com/LLMQuant/quant-mind/issues/71)). PR5 lands the
> apex layer (`flows/` + `magic.py`); the remaining work is the `mind/`
> memory + store layer scheduled for PR6 and PR7.
Paper Flow V1 is source-first: code preserves the exact PDF revision and
page-aware chunks before accepting a bounded, cited model summary. See the
[complete persist/reopen/search example](examples/flows/paper.py) and
[design contract](contexts/design/flow/paper.md).

---

Expand All @@ -238,7 +239,7 @@ asyncio.run(main())
- [x] Better `flow` design for user-friendly usage
- [x] First production level example (Quant Paper Agent)
- [ ] Migrate Agent layer to OpenAI Agents SDK
- [ ] Standardize knowledge format with `knowledge/` (Pydantic-based)
- [x] Standardize knowledge format with `knowledge/` (Pydantic-based)
- [ ] Additional content sources (financial news, blogs, reports)
- [ ] Cross-step working memory (`mind/memory`) for batch document processing

Expand All @@ -254,18 +255,10 @@ QuantMind is designed with a larger vision: to become a comprehensive intelligen
The foundation we're building today—starting with papers—will expand to encompass the entire financial information ecosystem.

> [!NOTE]
> **Future Conceptual Example (PR6 brings `FilesystemMemory`):**
>
> ```python
> from quantmind.configs.paper import ArxivIdentifier
> from quantmind.flows import paper_flow
> from quantmind.knowledge import Paper
> from quantmind.mind.memory import FilesystemMemory # PR6
>
> memory = FilesystemMemory("./mem/factor-research/")
> for arxiv_id in arxiv_ids:
> paper: Paper = await paper_flow(ArxivIdentifier(id=arxiv_id), memory=memory)
> ```
> The current source-first paper path produces independently versioned source,
> chunk-set, and cited-summary artifacts. Future agent memory and cross-document
> reasoning can build on `LocalKnowledgeLibrary.search()` without changing
> those canonical artifacts.

This future state represents our commitment to moving beyond simple data aggregation and toward genuine machine intelligence in the financial domain.

Expand Down
4 changes: 3 additions & 1 deletion contexts/design/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,14 @@ implementation must preserve.

| Domain | Design |
|---|---|
| Flow | [Paper extraction from input to validated result](flow/paper.md) |
| Flow | [Source-first paper flow](flow/paper.md) |
| Knowledge | [Paper sources, artifacts, citations, and locators](knowledge/paper.md) |
| Flow | [News collection](flow/news.md) |
| Preprocess | [Page-aware multimodal PDF parsing](preprocess/pdf.md) |
| RAG | [Page-aware document chunking and retrieval](rag/document.md) |
| Library | [Local knowledge storage and meaning-based search](library/local.md) |
| Operations | [Public operation naming](operations/naming.md) |
| Operations | [Orchestration and construction altitude](operations/orchestration.md) |

## Organization Rules

Expand Down
Loading
Loading