Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Default owner for everything — update as maintainers join.
* @Neal006

# High-blast-radius areas: benchmark correctness and published numbers.
/memorylens/evaluation/ @Neal006
/README.md @Neal006
/.github/ @Neal006
8 changes: 8 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: 💬 Questions & Discussions
url: https://github.com/Neal006/memorylens/discussions
about: Ask questions, share benchmark results, or discuss memory architectures.
- name: 🔒 Report a security vulnerability
url: https://github.com/Neal006/memorylens/security/advisories/new
about: Please report security issues privately, not as public issues.
19 changes: 19 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version: 2
updates:
- package-ecosystem: pip
directory: /
schedule:
interval: weekly
groups:
python-deps:
patterns: ["*"]
labels: ["area: dependencies"]

- package-ecosystem: github-actions
directory: /
schedule:
interval: monthly
groups:
actions:
patterns: ["*"]
labels: ["area: ci"]
49 changes: 49 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
"area: memory-backend":
- changed-files:
- any-glob-to-any-file: "memorylens/memory/**"

"area: evaluation":
- changed-files:
- any-glob-to-any-file: "memorylens/evaluation/**"

"area: simulator":
- changed-files:
- any-glob-to-any-file: "memorylens/simulator/**"

"area: providers":
- changed-files:
- any-glob-to-any-file: "memorylens/utils/providers.py"

"area: api":
- changed-files:
- any-glob-to-any-file: "memorylens/api.py"

"area: cli":
- changed-files:
- any-glob-to-any-file:
- "memorylens/cli.py"
- "main.py"

"area: dashboard":
- changed-files:
- any-glob-to-any-file: "dashboard.py"

"area: documentation":
- changed-files:
- any-glob-to-any-file:
- "**/*.md"
- "docs/**"

"area: tests":
- changed-files:
- any-glob-to-any-file: "tests/**"

"area: ci":
- changed-files:
- any-glob-to-any-file: ".github/**"

"area: dependencies":
- changed-files:
- any-glob-to-any-file:
- "pyproject.toml"
- "requirements.txt"
6 changes: 2 additions & 4 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ pytest tests/ -v
- [ ] Type hints used on all new function signatures
- [ ] No API key required to run any new tests
- [ ] If adding a backend: registered in `VALID_BACKENDS` and `_make_memory()` in `benchmark.py`
- [ ] If adding a scenario: `--scenario` flag added to `main.py`
- [ ] If adding a CLI flag: docstring example in `main.py` updated
- [ ] If adding a scenario: registered in the `SCENARIOS` dict in `simulator/scenarios/__init__.py`
- [ ] Lint passes: `ruff check .`
- [ ] README updated if there are new user-facing features or CLI flags
- [ ] No hardcoded API keys or secrets in any file

Expand All @@ -51,5 +51,3 @@ pytest tests/ -v
| Recall@T=100 | | |
| Tokens/query@T=100 | | |
| Cascade Efficiency | | |

🤖 Generated with [Claude Code](https://claude.ai/claude-code)
16 changes: 16 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,23 @@ on:
pull_request:
branches: [main]

permissions:
contents: read

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pipx run ruff check .
Comment on lines +20 to +24

test:
strategy:
fail-fast: false
Expand Down
23 changes: 23 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: CodeQL

on:
push:
branches: [main]
pull_request:
branches: [main]
schedule:
- cron: "31 5 * * 1"

permissions:
security-events: write
contents: read

jobs:
analyze:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: github/codeql-action/init@v3
with:
languages: python
- uses: github/codeql-action/analyze@v3
29 changes: 29 additions & 0 deletions .github/workflows/greetings.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Welcome

on:
issues:
types: [opened]
pull_request_target:
types: [opened]

permissions:
issues: write
pull-requests: write

jobs:
greet:
runs-on: ubuntu-latest
steps:
- uses: actions/first-interaction@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: >
Thanks for opening your first MemoryLens issue! A maintainer will triage it soon.
If you'd like to work on it yourself, comment here to claim it —
[CONTRIBUTING.md](https://github.com/Neal006/memorylens/blob/main/CONTRIBUTING.md)
has everything you need to get a dev environment running in 5 minutes.
pr-message: >
Thanks for your first contribution to MemoryLens! 🎉 CI will run the test suite
on Linux/macOS/Windows across Python 3.10–3.13 — no API key needed.
A maintainer will review within 48 hours. If CI fails, `pytest tests/ -v` and
`ruff check .` locally reproduce almost every failure.
16 changes: 16 additions & 0 deletions .github/workflows/labeler.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Label PRs

on:
pull_request_target:

permissions:
contents: read
pull-requests: write

jobs:
label:
runs-on: ubuntu-latest
steps:
- uses: actions/labeler@v5
with:
sync-labels: true
28 changes: 28 additions & 0 deletions .github/workflows/pr-title.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: PR Title

on:
pull_request:
types: [opened, edited, synchronize, reopened]

permissions:
pull-requests: read

jobs:
lint:
runs-on: ubuntu-latest
steps:
# PRs are squash-merged, so the PR title becomes the commit message on main.
- uses: amannn/action-semantic-pull-request@v5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
types: |
feat
fix
perf
refactor
test
docs
chore
ci
style
32 changes: 32 additions & 0 deletions .github/workflows/stale.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Stale

on:
schedule:
- cron: "17 3 * * *"
workflow_dispatch:

permissions:
issues: write
pull-requests: write

jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v9
with:
days-before-issue-stale: 60
days-before-issue-close: 14
days-before-pr-stale: 30
days-before-pr-close: 14
stale-issue-label: "status: stale"
stale-pr-label: "status: stale"
exempt-issue-labels: "status: open-for-contribution,pinned,roadmap,help wanted,good first issue"
exempt-pr-labels: "pinned"
stale-issue-message: >
This issue has been inactive for 60 days. It will close in 14 days
unless there is new activity. Comment or add the `pinned` label to keep it open.
stale-pr-message: >
This PR has been inactive for 30 days. It will close in 14 days
unless there is new activity. Push a commit or comment to keep it open.
operations-per-run: 100
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,5 @@ demo_venv/
benchmark_v04_200.json
*.db
experiment_logs/runs_summary*.csv
.pytest_cache/
.ruff_cache/
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ We pledge to act and interact in ways that contribute to an open, welcoming, div

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening a GitHub issue or contacting the maintainer directly. All complaints will be reviewed and investigated promptly and fairly.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported privately to the maintainer at builtbyneal@gmail.com. All complaints will be reviewed and investigated promptly and fairly, and reporter confidentiality will be respected.

## Attribution

Expand Down
11 changes: 9 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,10 +286,17 @@ refactor: extract _extract_entity() helper from EntityMemory

1. Fork the repo and create a branch: `git checkout -b feat/your-feature`
2. Make your changes with tests
3. Run `pytest tests/ -v` — all tests must pass
3. Run `pytest tests/ -v` and `ruff check .` — both must pass
4. Open a PR against `main` — fill in the PR template
5. Reference the issue: `Closes #<issue-number>` in the PR description

**What happens automatically when you open a PR:**

- CI runs lint + the full test suite on Linux, macOS, and Windows (Python 3.10–3.13) and builds the package
- Your **PR title is checked** against Conventional Commits (`feat: ...`, `fix: ...`) — PRs are squash-merged, so the title becomes the commit message
- Area labels (`area: memory-backend`, `area: documentation`, …) are applied from the files you changed
- PRs inactive for 30 days are marked stale and closed after 14 more — push a commit or comment to keep one alive

**PR checklist:**
- [ ] All existing tests pass: `pytest tests/ -v`
- [ ] New tests added for new functionality
Expand All @@ -311,7 +318,7 @@ refactor: extract _extract_entity() helper from EntityMemory
| **No new top-level dependencies** without issue discussion | Keeps install size predictable |
| **All new metrics return `float` in `[0, 1]`** | Ensures dashboard and aggregation code work without guards |
| **All tests pass without an API key** | Keeps CI fast and accessible to all contributors |
| **PEP 8**, 100-char line limit | Consistency |
| **`ruff check .` passes** (PEP 8, 100-char lines) | Enforced in CI — no style debates in review |

---

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,7 @@ pip install -e ".[server,dev]"
pytest tests -q # all green before you start
```

Start here: [`good first issue`](https://github.com/Neal006/memorylens/issues?q=label%3A%22good+first+issue%22) · Guide: [CONTRIBUTING.md](CONTRIBUTING.md) · Plans: [ROADMAP.md](ROADMAP.md)
Start here: [`good first issue`](https://github.com/Neal006/memorylens/issues?q=label%3A%22good+first+issue%22) · Guide: [CONTRIBUTING.md](CONTRIBUTING.md) · Plans: [ROADMAP.md](ROADMAP.md) · Questions: [Discussions](https://github.com/Neal006/memorylens/discussions) · Vulnerabilities: [SECURITY.md](SECURITY.md)

---

Expand Down
23 changes: 23 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Security Policy

## Supported Versions

Only the latest release on [PyPI](https://pypi.org/project/memorylens-bench/) receives security fixes.

## Reporting a Vulnerability

Please **do not open a public issue** for security problems.

Report privately via [GitHub Security Advisories](https://github.com/Neal006/memorylens/security/advisories/new)
or email builtbyneal@gmail.com.

You can expect an acknowledgement within 72 hours and a fix or mitigation plan within 14 days
for confirmed issues.

## Scope notes

MemoryLens is a local benchmark tool. The most security-relevant surfaces are:

- The optional FastAPI server (`memorylens.api`) — intended for local use, ships with no auth.
Do not expose it to the public internet without a reverse proxy handling auth and rate limits.
- LLM provider keys read from `.env` — never committed, never logged.
6 changes: 3 additions & 3 deletions dashboard.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

import json
import os
from typing import Dict, List, Optional
from typing import Dict, List

import pandas as pd
import plotly.graph_objects as go
Expand Down Expand Up @@ -235,8 +235,8 @@ def render_results(data: Dict, is_demo: bool = False) -> None:
content_vals = data[name]["recall"]
llm_vals = data[name].get("llm_recall", [None] * len(content_vals))
gaps = [
(c - l) * 100 if l is not None else None
for c, l in zip(content_vals, llm_vals)
(c - lv) * 100 if lv is not None else None
for c, lv in zip(content_vals, llm_vals)
]
if any(g is not None for g in gaps):
fig_gap.add_trace(go.Bar(
Expand Down
2 changes: 1 addition & 1 deletion memorylens/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

import threading
import uuid
from typing import Dict, List, Optional
from typing import Dict, List

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
Expand Down
Loading
Loading