Make your AI give better answers — by giving it better standing instructions.
judge-dredd is a lightweight program that helps you write a pre-seed prompt: a block of instructions you paste into Claude, ChatGPT, or another AI account once, so that you can tune how your AI responds, and how it formats its answers, to something you prefer.
You can:
- Create a pre-seed prompt — start from a ready-made template, write your own, or just describe what you want and let the program draft one for you.
- Test it — see how it actually changes the AI's answers on sample questions, and score it (this step needs an "API key" — see below).
- Export it — save it as a file, then copy-paste it into your AI account.
You do not need to know how to code to use this. The guide below assumes you have never opened a "terminal" in your life.
- A Windows PC or a Mac.
- About 10 minutes for the one-time setup.
- (Only for the "Test" feature) an Anthropic API key (for now) — see About the API key. You can create and export prompts without a key.
You'll do steps 1–4 once. After that, starting the program is just one command.
"Node.js" is the free engine that runs this program. Install it once:
- Go to https://nodejs.org
- Click the big button that says "LTS" (it stands for the stable version).
- Open the file you downloaded and click Next → Next → Install (the defaults are fine).
- When it finishes, you're done. You won't see an app icon — Node.js works behind the scenes.
If you were given a .zip file: right-click it and choose Extract All (Windows) or
double-click it (Mac). Remember the folder it creates — for example Downloads\judge-dredd.
If you know how to use Git, you can instead clone the repository. Either way, you end up with a
folder containing run.js and the other files.
A "terminal" is just a window where you type commands. Here's the easy way to open one already pointed at the right folder:
On Windows:
- Open the
judge-dreddfolder in File Explorer. - Click once in the address bar at the top (where the folder path is shown).
- Type
powershelland press Enter. A blue/black window opens. That's the terminal.
On Mac:
- Open the
judge-dreddfolder in Finder. - Right-click the folder and choose New Terminal at Folder.
(If you don't see that option: open the Terminal app, type
cd— with a space — then drag the folder into the window, and press Enter.)
In that terminal window, type this and press Enter:
npm install
It will think for a few seconds and print some lines. When you get a normal prompt back, it's done. You only ever need to do this once.
Type this and press Enter:
npm start
The program opens with a menu. Success! From now on, steps 3 and 5 are all you need to run judge-dredd:
open a terminal in the folder, type npm start. :~)
When it starts you'll see a menu. Type the number (or letter) of what you want and press Enter.
- 1 — Create a pre-seed prompt. Pick a template, paste your own text, or describe your goal
and let "Dredd" draft one. When pasting your own, finish by typing a single dot
.on its own line. - 2 — Test a prompt. Runs your prompt on a set of sample questions so you can see the difference. You can score one prompt, or put two head-to-head and pick the better (blind). The two answers are shown one after the other by default — you can switch to side-by-side in Settings. This needs an "API key".
- 3 — Export a prompt to a file. Saves it into an
exportsfolder as a.mdor.txtfile. - 4 — Settings. Choose which AI models the Test feature uses (one to answer, one to judge), how many sample questions each test runs, and whether two answers are compared stacked (the default, most readable) or side-by-side.
- H — Help, Q — Quit.
Open the file from the exports folder, copy all the text, and paste it into your AI account's
custom instructions (ChatGPT) or project / personal preferences (Claude). From then on,
that AI will follow your instructions in every chat. I disclaim responsibility if your AI gains
sentience by use of this program.
You only need a key for the Test feature (and if you ask Dredd to draft a prompt "online"). Creating and exporting prompts works with no key at all.
- The key is an Anthropic API key from https://console.anthropic.com. It is a pay-as-you-go developer key (you're billed a small amount per test).
- It is not the same as a Claude.ai Pro or Max subscription. A normal Claude.ai subscription cannot be used to run this program — only a Console API key can.
- judge-dredd never saves your key. It reads it from an environment variable called
ANTHROPIC_API_KEYif you've set one, otherwise it asks you to paste it (and hides it as you type). It is kept in memory only, for that one run. - Note that if you purchase an API key, that key is unique to you and will only be shown to you once. Keep it safe and secure (if not, bad people would take your key and spend your credits).
- I hope to add OpenAI capability soon.
To avoid pasting it every time, you can set the environment variable once per terminal session:
- Windows (PowerShell):
$env:ANTHROPIC_API_KEY = "your-key-here"thennpm start - Mac:
export ANTHROPIC_API_KEY="your-key-here"thennpm start
You can edit any of these in a text editor to tune the program — nothing breaks if you just read them:
prompts.js— the pre-seed prompt templates (the starting points).samples.js— the sample questions used when testing a prompt.judge.js— the rubric: what the built-in judge considers a "good" answer.run.js— the program itself (the menu).prompt.js,config.js,generate.js,evaluate.js,export.js,ui.js— supporting parts.
Each test run saves a report-<number>.json file with the full questions, answers, and scores,
so you can re-read them later.
- "npm is not recognized" / "command not found: npm" — Node.js isn't installed, or the terminal was open before you installed it. Close the terminal, reopen it (Step 3), and try again.
- The menu shows odd symbols instead of the title — your terminal's font doesn't support the block letters. It's cosmetic; everything still works.
- "Testing needs an Anthropic API key" — that feature needs a key (see above). Create and export still work without one.
- "API key rejected (401)" — the key is wrong, expired, or has a typo. Get a fresh one from console.anthropic.com.
- Something went wrong during a test (e.g. a network blip) — the program shows a short message and drops you back at the menu; it won't close on you. Just pick the option again.
- A comparison looks cramped or wrapped oddly — that's the side-by-side view in a narrow window. Widen the terminal, or set the comparison layout back to stacked in Settings.
- Review injected queries - scope to specific jurisdiction (currently common law general; has scope to include major civil law jurisdictions). Current ones are really general, so as the prompt is iterated, Dredd should default to more difficult questions. Scaffold for difficulty rating but weigh against knowledge/context limitations from an online search.
- OpenAI API key support.
- Larger test pool.
- API key usage warning.