Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
e857983
feat: persist config in sqlite instead of config.env
whoisjayd Sep 13, 2026
1d65513
fix: harden pdf export with bundled unicode font
whoisjayd Sep 13, 2026
dbe1029
fix: llm provider usage accounting and error redaction
whoisjayd Sep 13, 2026
ccfe04f
fix: escape prompt injection in transcript prompts
whoisjayd Sep 13, 2026
6ad5d0d
fix: close silent exception handling and rate limiter leak
whoisjayd Sep 13, 2026
a074185
refactor: consolidate constants and remove duplicate logic
whoisjayd Sep 13, 2026
44f6daa
build: switch release binary to onedir packaging
whoisjayd Sep 13, 2026
451ee22
fix: silence bandit sql-injection false positives in config store
whoisjayd Sep 13, 2026
e92e4a6
fix: never follow a symlink planted at the config db path
whoisjayd Sep 13, 2026
0765cf4
fix: assert symlink-swap test via config api not raw text read
whoisjayd Sep 13, 2026
c198265
chore(release): v1.6.0
whoisjayd Sep 13, 2026
3ac63e6
fix: evict loop-scoped youtube limiters on close instead of relying o…
whoisjayd Sep 13, 2026
99875f7
fix: strip signed query params from urls before they reach logs
whoisjayd Sep 13, 2026
887c58f
fix: escape second-stage llm fragments in stitch and quiz prompts
whoisjayd Sep 13, 2026
80bdc6b
docs: fix stale pdf capability and config precedence wording
whoisjayd Sep 13, 2026
c78c6e6
chore: trim trailing whitespace in bundled font license
whoisjayd Sep 13, 2026
4a4f33a
fix: harden config storage against symlink races and unvalidated writes
whoisjayd Sep 13, 2026
0aa20dd
chore: ignore playwright mcp output directory
whoisjayd Sep 13, 2026
4a5d920
chore(deps): update dependencies and bump litellm to 1.100.1
whoisjayd Sep 13, 2026
c895290
fix: expand setup wizard model catalog to stop missing valid provider…
whoisjayd Sep 13, 2026
3ccf699
feat: redesign homepage copy, motion, and provider coverage
whoisjayd Sep 13, 2026
7a03544
docs: sync provider credential list into llms aggregate files
whoisjayd Sep 13, 2026
6652c24
fix: warn against following instructions inside quiz_section tags
whoisjayd Sep 13, 2026
65baeb7
fix: fail closed when config directory permissions cannot be hardened
whoisjayd Sep 13, 2026
82c4e0f
fix: scope dev server fs access to repo root and scripts only
whoisjayd Sep 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# Example notewise runtime configuration
#
# This project reads configuration from:
# ~/.notewise/config.env
# ~/.notewise/config.db (managed via `notewise setup`/`notewise config`)
#
# The repository root .env file is not read automatically by the application.
# Copy the keys you need into ~/.notewise/config.env or export them from your shell.
# Set the keys you need with `notewise config set KEY VALUE`, run `notewise
# setup`, or export them from your shell.

DEFAULT_MODEL=gemini/gemini-2.5-flash
# Allow models missing from the bundled supported-model catalog (e.g. OpenRouter stealth endpoints).
Expand Down Expand Up @@ -50,9 +51,10 @@ GEMINI_API_KEY=your_gemini_api_key_here
# JINA_API_KEY=your_jina_api_key_here

# Custom OpenAI-compatible endpoint registry
# Run `notewise setup` to add an endpoint, discover its models, and verify a selection.
# Store every profile in one compact JSON array. This value contains credentials; use
# placeholder values only in shared files and keep real values in ~/.notewise/config.env.
# Manage saved endpoints with `notewise inference list|add|update|delete` -- each
# is stored as its own row in ~/.notewise/config.db, not as an env var. The
# CUSTOM_LLM_ENDPOINTS variable below only overrides the whole registry for one
# process (a compact JSON array); use placeholder values only in shared files.
# CUSTOM_LLM_ENDPOINTS='[{"name":"team-gateway","base_url":"https://gateway.example.com/v1","api_key":"example-team-token"},{"name":"lab-server","base_url":"https://lab.example.net/v1","api_key":"example-lab-token"}]'
# Custom models are always named <name>/<model-id>.
# DEFAULT_MODEL=team-gateway/your-model-id
Expand All @@ -70,4 +72,4 @@ GEMINI_API_KEY=your_gemini_api_key_here
# Notes
# - The current config loader supports the keys above.
# - default_languages, chunk_size, and chunk_overlap are code defaults today,
# not config.env keys.
# not config.db keys.
11 changes: 6 additions & 5 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ jobs:
uv run pyinstaller `
--noconfirm `
--clean `
--onefile `
--onedir `
--name notewise `
--collect-all litellm `
--collect-all tiktoken `
Expand All @@ -119,17 +119,18 @@ jobs:
$artifactName = "${{ matrix.artifact_name }}"
$archiveExt = "${{ matrix.archive_ext }}"
$archName = "${{ runner.arch }}".ToLowerInvariant()
$binaryName = if ("${{ runner.os }}" -eq "Windows") { "notewise.exe" } else { "notewise" }
$archiveBase = "notewise-$releaseTag-$artifactName-$archName"

# PyInstaller --onedir emits dist/notewise/ containing the entry
# binary plus its _internal/ support files; the whole directory
# must ship together so the binary can find its dependencies.
New-Item -ItemType Directory -Path release -Force | Out-Null
Copy-Item "dist/$binaryName" "release/$binaryName"

if ($archiveExt -eq "zip") {
Compress-Archive -Path "release/$binaryName" -DestinationPath "release/$archiveBase.zip" -Force
Compress-Archive -Path "dist/notewise/*" -DestinationPath "release/$archiveBase.zip" -Force
}
else {
tar -czf "release/$archiveBase.tar.gz" -C release $binaryName
tar -czf "release/$archiveBase.tar.gz" -C "dist/notewise" .
}
- name: Upload binary artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
Expand Down
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ repos:
- id: check-executables-have-shebangs # scripts without shebangs fail silently
- id: check-shebang-scripts-are-executable # inverse: shebang ⟹ must be +x
- id: check-added-large-files
exclude: ^src/notewise/ui/fonts/.*\.ttf$ # bundled Noto Sans PDF fonts
- id: debug-statements
- id: detect-private-key
- id: end-of-file-fixer
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ src/notewise/
├── ui/oauth_flow.py ← LiteLLM OAuth/device-flow login helpers
├── ui/setup_wizard.py ← Interactive provider/config setup wizard
├── ui/litellm_models_snapshot.json ← Bundled text-only LiteLLM setup catalog
├── ui/fonts/ ← Bundled Noto Sans TTFs used by pipeline/_documents.py PDF export
└── youtube/ ← Transcript & metadata extraction; no LLM calls here
```

Expand Down
48 changes: 34 additions & 14 deletions docs/config/configuration.mdx
Original file line number Diff line number Diff line change
@@ -1,28 +1,34 @@
---
title: "Configuration"
sidebarTitle: "Configuration"
description: "Understand NoteWise config files, environment variables, defaults, and command overrides."
keywords: ["notewise config", "config.env", "NOTEWISE_HOME", "DEFAULT_MODEL"]
description: "Understand NoteWise config storage, environment variables, defaults, and command overrides."
keywords: ["notewise config", "config.db", "NOTEWISE_HOME", "DEFAULT_MODEL"]
---

NoteWise reads settings from code defaults, `config.env`, environment variables, and command flags.
NoteWise reads settings from code defaults, `config.db`, environment variables, and command flags.

## State directory

By default, state lives under `~/.notewise`.

| File or folder | Purpose |
| -------------------------------- | ---------------------------------------- |
| `~/.notewise/config.env` | User config written by `notewise setup`. |
| `~/.notewise/config.db` | User config written by `notewise setup`/`config`. |
| `~/.notewise/.notewise_cache.db` | SQLite cache and run history. |
| `~/.notewise/logs/` | Session logs. |
| `~/.notewise/oauth/` | Default OAuth token storage. |

Set `NOTEWISE_HOME` to move this state root.

<Note>
Older installs stored config in a `~/.notewise/config.env` text file. On the
first run after upgrading, NoteWise imports any existing `config.env` into
`config.db` and deletes the file — no manual migration needed.
</Note>

## Practical precedence

For a command run, command-line flags win. For `OUTPUT_DIR`, the value in `config.env` is used unless `--output` is passed. Other settings use environment variables, then `config.env`, then code defaults.
For a command run, command-line flags win. For `OUTPUT_DIR`, the value in `config.db` is used unless `--output` is passed. Other settings use environment variables, then `config.db`, then code defaults.
Comment thread
whoisjayd marked this conversation as resolved.

```bash
notewise process "https://youtu.be/VIDEO_ID" --model gemini/gemini-2.5-flash --output ./notes
Expand All @@ -44,13 +50,15 @@ cannot use LiteLLM provider prefixes. Base URLs must be absolute HTTPS URLs, exc
explicit loopback endpoints may use HTTP. Setup accepts a URL with or without `/v1`
and stores it with the `/v1` suffix.

The registry is one compact JSON value in `~/.notewise/config.env`. Each object has
`name`, `base_url`, and `api_key` fields. This safe two-profile example uses
placeholder credentials:
The registry is stored as one row per endpoint (`name`, `base_url`, `api_key`) in a
dedicated table in `~/.notewise/config.db`, managed with `notewise inference
list|add|update|delete`. You can also override the whole registry for one process
with the `CUSTOM_LLM_ENDPOINTS` environment variable, a compact JSON array with the
same fields — this safe two-profile example uses placeholder credentials:

```dotenv
CUSTOM_LLM_ENDPOINTS='[{"name":"team-gateway","base_url":"https://gateway.example.com/v1","api_key":"example-team-token"},{"name":"lab-server","base_url":"https://lab.example.net/v1","api_key":"example-lab-token"}]'
DEFAULT_MODEL=team-gateway/your-model-id
```bash
export CUSTOM_LLM_ENDPOINTS='[{"name":"team-gateway","base_url":"https://gateway.example.com/v1","api_key":"example-team-token"},{"name":"lab-server","base_url":"https://lab.example.net/v1","api_key":"example-lab-token"}]'
export DEFAULT_MODEL=team-gateway/your-model-id
```

Custom models are always named `<name>/<model-id>`. When that name exactly matches
Expand All @@ -73,11 +81,11 @@ copies its credentials into ambient environment variables.
| `YOUTUBE_COOKIE_FILE` | unset | Default Netscape cookies file path. |
| `ALLOW_UNLISTED_MODELS` | `false` | Permit out-of-catalog models. |

Provider API keys and provider auth keys are also accepted in `config.env` when they are listed in source constants.
Provider API keys and provider auth keys are also accepted in `config.db` (via `notewise config set KEY VALUE`) when they are listed in source constants.

<Note>
`chunk_size`, `chunk_overlap`, and `max_concurrent_chapters` are code-default
settings, not normal `config.env` keys.
settings, not normal `config.db` keys.
</Note>

## Commands
Expand All @@ -87,8 +95,20 @@ notewise setup
notewise setup --show
notewise setup --force
notewise config
notewise config keys
notewise config get DEFAULT_MODEL
notewise config set DEFAULT_MODEL gemini/gemini-2.5-flash
notewise config unset MAX_TOKENS
notewise config-path
notewise edit-config
```

`notewise config` masks secrets before printing settings.
`notewise config` (no arguments) masks secrets before printing all settings.
`notewise config keys` lists every key `get`/`set`/`unset` accept, since the
full set (provider API keys, provider auth keys, and core settings) isn't
obvious up front. `notewise config get/set/unset` read or write one key at a
time; `set` validates the new value immediately and reports an error without
leaving the process in a broken state if it's invalid (e.g. `TEMPERATURE` out
of range). `edit-config`
opens the current settings as an editable env-style file in your `$EDITOR`, then
re-saves whatever you leave in it — including removing lines you delete.
2 changes: 1 addition & 1 deletion docs/config/oauth.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: "Sign in to ChatGPT or GitHub Copilot through LiteLLM OAuth/device-
keywords: ["notewise auth login", "chatgpt", "github copilot", "oauth", "codex"]
---

ChatGPT subscription and GitHub Copilot use LiteLLM OAuth/device-flow login. They do not use static API keys in `config.env`. User configuration is stored at `~/.notewise/config.env`; do not create a repo-root `.env` file for OAuth setup.
ChatGPT subscription and GitHub Copilot use LiteLLM OAuth/device-flow login. They do not use static API keys in `config.db`. User configuration is stored at `~/.notewise/config.db`; do not create a repo-root `.env` file for OAuth setup.

## Login commands

Expand Down
16 changes: 8 additions & 8 deletions docs/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ Create config:
notewise setup
```

The setup wizard writes supported keys to `~/.notewise/config.env` unless `NOTEWISE_HOME` points somewhere else.
The setup wizard writes supported keys to `~/.notewise/config.db` unless `NOTEWISE_HOME` points somewhere else.

Process one public YouTube video:

Expand Down Expand Up @@ -298,20 +298,20 @@ Persist `/home/notewise/.notewise` if you want cache, logs, config, and OAuth to

## Configuration

NoteWise reads settings from code defaults, `config.env`, environment variables, and command flags.
NoteWise reads settings from code defaults, `config.db`, environment variables, and command flags.

By default, state lives under `~/.notewise`.

| File or folder | Purpose |
| --- | --- |
| `~/.notewise/config.env` | User config written by `notewise setup`. |
| `~/.notewise/config.db` | User config written by `notewise setup`. |
| `~/.notewise/.notewise_cache.db` | SQLite cache and run history. |
| `~/.notewise/logs/` | Session logs. |
| `~/.notewise/oauth/` | Default OAuth token storage. |

Set `NOTEWISE_HOME` to move the state root.

For a command run, command-line flags win. Then environment variables, then `config.env`, then code defaults.
For a command run, command-line flags win. Then environment variables, then `config.db`, then code defaults.

```bash
notewise process "https://youtu.be/VIDEO_ID" --model gemini/gemini-2.5-flash --output ./notes
Expand All @@ -330,9 +330,9 @@ Common config keys:
| `MAX_TOKENS` | unset | Optional LLM max output tokens. |
| `YOUTUBE_COOKIE_FILE` | unset | Default Netscape cookies file path. |

Provider API keys and provider auth keys are also accepted in `config.env` when they are listed in source constants.
Provider API keys and provider auth keys are also accepted in `config.db` when they are listed in source constants.

`chunk_size`, `chunk_overlap`, and `max_concurrent_chapters` are code-default settings, not normal `config.env` keys.
`chunk_size`, `chunk_overlap`, and `max_concurrent_chapters` are code-default settings, not normal `config.db` keys.

Saved custom endpoints use `<name>/<model-id>`. `notewise process --model <name>/<model-id>` automatically selects that profile's base URL and API key, then sends LiteLLM `openai/<model-id>`. Custom names cannot use LiteLLM provider prefixes; remote endpoints require HTTPS; and changing `--base-url` requires a new `--api-key` instead of reusing a saved credential. Manage profiles with `notewise inference list|add|update|delete`; never expose the secret-bearing `CUSTOM_LLM_ENDPOINTS` value.

Expand Down Expand Up @@ -394,7 +394,7 @@ When a model string has no explicit provider prefix, NoteWise can infer common k

## OAuth login

ChatGPT subscription and GitHub Copilot use LiteLLM OAuth/device-flow login. They do not use static API keys in `config.env`. User configuration is stored at `~/.notewise/config.env`; do not create a repo-root `.env` file for OAuth setup.
ChatGPT subscription and GitHub Copilot use LiteLLM OAuth/device-flow login. They do not use static API keys in `config.db`. User configuration is stored at `~/.notewise/config.db`; do not create a repo-root `.env` file for OAuth setup.

Login commands:

Expand Down Expand Up @@ -599,7 +599,7 @@ Output formats:
| --- | --- |
| `md` | Raw Markdown notes. Default. |
| `html` | Markdown rendered with built-in CSS. |
| `pdf` | Uses `fpdf`; Latin-script text only. Unsupported Unicode falls back to Markdown. |
| `pdf` | Uses `fpdf2` with a bundled Noto Sans font (Latin, Cyrillic, Greek, Vietnamese, Devanagari). Other scripts (e.g. Arabic, Hebrew, CJK), or any renderer failure, fall back to Markdown for that video instead of failing the run. |
| `docx` | Uses `html2docx` and `python-docx`. |

Example output layout:
Expand Down
2 changes: 1 addition & 1 deletion docs/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Important `process` flags:

Config truth:

- Config file: `~/.notewise/config.env`
- Config file: `~/.notewise/config.db`
- `NOTEWISE_HOME` changes the state root.
- Practical precedence: CLI flags, then environment variables, then config file, then defaults.
- Common keys: `DEFAULT_MODEL`, `CUSTOM_LLM_ENDPOINTS`, `OUTPUT_DIR`, `MAX_CONCURRENT_VIDEOS`, `YOUTUBE_REQUESTS_PER_MINUTE`, `TEMPERATURE`, `MAX_TOKENS`, `YOUTUBE_COOKIE_FILE`, provider API/auth keys.
Expand Down
10 changes: 7 additions & 3 deletions docs/operate/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,12 @@ Run `notewise COMMAND --help` for command-specific flags. The command set is def
| `notewise process URL` | Generate notes from one video, playlist, or batch file. |
| `notewise setup` | Run the interactive config wizard. |
| `notewise config` | Print masked effective config. |
| `notewise config-path` | Print the active config path. |
| `notewise edit-config` | Open the config file in an editor. |
| `notewise config keys` | List every key `get`/`set`/`unset` accept. |
| `notewise config get KEY` | Print one config value (masked if secret). |
| `notewise config set KEY VAL` | Set one config value. |
| `notewise config unset KEY` | Remove one config value. |
| `notewise config-path` | Print the active config database path. |
| `notewise edit-config` | Edit config as an env-style file in your editor. |
| `notewise auth login PROVIDER` | Login to OAuth providers. |
| `notewise info` | Show runtime config and state paths. |
| `notewise info URL` | Inspect YouTube metadata and cache status without generation. |
Expand Down Expand Up @@ -48,7 +52,7 @@ to a different OpenAI-compatible URL:
notewise process "<video-url>" --model <name>/<model-id> --base-url <base-url> --api-key <api-key>
```

Both flags are transient: they do not update `config.env` or ambient environment
Both flags are transient: they do not update `config.db` or ambient environment
variables. `--base-url` takes precedence for the request URL. An explicitly passed
`--api-key` takes precedence for the request key; otherwise, NoteWise reuses a
saved profile key only when the normalized URL is the profile's saved origin.
Expand Down
2 changes: 1 addition & 1 deletion docs/skill.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Commands: `process`, `setup`, `config`, `config-path`, `version`, `update`, `sta

## Config

Config file: `~/.notewise/config.env`. `NOTEWISE_HOME` changes the state root. Practical precedence: CLI flags, then environment variables, then config file, then defaults. Exception: `OUTPUT_DIR` from config file is respected unless `--output/-o` is passed.
Config database: `~/.notewise/config.db`. `NOTEWISE_HOME` changes the state root. Practical precedence: CLI flags, then environment variables, then config database, then defaults. Exception: `OUTPUT_DIR` from the config database is respected unless `--output/-o` is passed.

Common keys: `DEFAULT_MODEL`, `CUSTOM_LLM_ENDPOINTS`, `OUTPUT_DIR`, `MAX_CONCURRENT_VIDEOS`, `YOUTUBE_REQUESTS_PER_MINUTE`, `TEMPERATURE`, `MAX_TOKENS`, `YOUTUBE_COOKIE_FILE`, provider API/auth keys. Custom models use `<name>/<model-id>` and `CUSTOM_LLM_ENDPOINTS` contains their API keys, so it must remain masked. Do not present `chunk_size`, `chunk_overlap`, or `max_concurrent_chapters` as normal config-file keys.

Expand Down
2 changes: 1 addition & 1 deletion docs/start/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Start here after installing NoteWise. This page covers the first successful run
notewise setup
```

The setup wizard writes supported keys to `~/.notewise/config.env` unless `NOTEWISE_HOME` points somewhere else.
The setup wizard writes supported keys to `~/.notewise/config.db` unless `NOTEWISE_HOME` points somewhere else.

</Step>
<Step title="Process one video">
Expand Down
2 changes: 1 addition & 1 deletion docs/understand/pipeline-output.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Videos with YouTube chapters use chapter-aware generation. Without `--chapter-di
| ------ | -------------------------------------------------------------------------------- |
| `md` | Raw Markdown notes. Default. |
| `html` | Markdown rendered with built-in CSS. |
| `pdf` | Uses `fpdf`; Latin-script text only. Unsupported Unicode falls back to Markdown. |
| `pdf` | Uses `fpdf2` with a bundled Noto Sans font (Latin, Cyrillic, Greek, Vietnamese, Devanagari). Other scripts (e.g. Arabic, Hebrew, CJK), or any renderer failure, fall back to Markdown for that video instead of failing the run. |
| `docx` | Uses `html2docx` and `python-docx`. |

## File layout
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ dependencies = [
"html2docx>=1.6.0",
"python-docx>=1.2.0",
"fpdf2>=2.8.8",
"fonttools>=4.62.1",
"defusedxml>=0.7.1",
]

Expand Down
4 changes: 3 additions & 1 deletion scripts/install.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,10 @@ try {
$extractDir = Join-Path $tempDir "extracted"
Expand-Archive -LiteralPath $archivePath -DestinationPath $extractDir -Force

# The release archive is a PyInstaller --onedir bundle: notewise.exe plus
# its _internal/ support files, which must stay together on disk.
New-Item -ItemType Directory -Path $installDir -Force | Out-Null
Copy-Item (Join-Path $extractDir "notewise.exe") (Join-Path $installDir "notewise.exe") -Force
Copy-Item (Join-Path $extractDir "*") $installDir -Recurse -Force

$userPath = [Environment]::GetEnvironmentVariable("Path", "User")
$pathEntries = @()
Expand Down
Loading
Loading