Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 37 additions & 7 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,11 +1,41 @@
# -----------------------------------------------------------------------------
# LLM Provider Configuration
# Options: "ollama" or "gemini"
LLM_PROVIDER=ollama
# Options: "ollama" (default), "gemini", "claude_code", or "codex"
# -----------------------------------------------------------------------------

# Default model to use
# For Ollama: "gemma3:4b", "qwen3:4b", "mistral:7b", etc.
# For Gemini: "gemini-2.5-pro", "gemini-2.5-flash", etc.
# Ollama mode - fully local models. Requires `ollama serve` and a pulled model.
LLM_PROVIDER=ollama
DEFAULT_MODEL=gemma3:4b

# Google Gemini API Key (required if using Gemini provider)
GEMINI_API_KEY=your_gemini_api_key_here
# -----------------------------------------------------------------------------
# Alternative: Claude Code CLI (no API key - uses your authenticated session)
# -----------------------------------------------------------------------------
# LLM_PROVIDER=claude_code
# DEFAULT_MODEL=claude-code

# Optional: command or full path to the Claude Code CLI.
# CLAUDE_CODE_COMMAND=claude

# Optional: per-call timeout in seconds.
# CLAUDE_CODE_TIMEOUT_SECONDS=300

# Optional: pin a specific model / alias for the nested calls, e.g. "sonnet",
# "opus", or a full id like "claude-sonnet-4-6". Leave unset to use your Claude
# Code session's default model. A cheaper/faster alias lowers cost per resume.
# CLAUDE_CODE_MODEL=sonnet

# -----------------------------------------------------------------------------
# Alternative: Codex CLI (no API key - uses your authenticated Codex session)
# -----------------------------------------------------------------------------
# LLM_PROVIDER=codex
# DEFAULT_MODEL=codex-cli
# CODEX_COMMAND=codex
# CODEX_TIMEOUT_SECONDS=300
# CODEX_MODEL=

# -----------------------------------------------------------------------------
# Alternative: Google Gemini (requires an API key)
# -----------------------------------------------------------------------------
# LLM_PROVIDER=gemini
# DEFAULT_MODEL=gemini-2.5-flash
# GEMINI_API_KEY=your_gemini_api_key_here
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
resume/*.pdf
run/*.pdf
test_*.py
!tests/
!tests/test_*.py
cache/
resume_evaluations.csv
greenhouse_resumes/*
Expand Down
111 changes: 103 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@

## Overview

Hiring Agent parses a resume PDF to Markdown, extracts sectioned JSON using a local or hosted LLM, augments the data with GitHub profile and repository signals, then produces an objective evaluation with category scores, evidence, bonus points, and deductions. You can run fully local with Ollama or use Google Gemini.
Hiring Agent parses a resume PDF to Markdown, extracts sectioned JSON using a local or hosted LLM, augments the data with GitHub profile and repository signals, then produces an objective evaluation with category scores, evidence, bonus points, and deductions. It defaults to fully local **Ollama** and can also run with **Google Gemini** or, with no API key, through an already-authenticated local **Claude Code CLI** or **Codex CLI**.

---

Expand Down Expand Up @@ -85,11 +85,16 @@ Hiring Agent parses a resume PDF to Markdown, extracts sectioned JSON using a lo

The repository pins `.python-version` to 3.11.13.

- **One LLM backend** (either of them)
- **One LLM backend** (any one of them)

- **Ollama** for local models
Install from the [official site](https://ollama.com/), then run `ollama serve`.
- **Google Gemini** if you have an API key, get it from [here](https://aistudio.google.com/api-keys).
- **Claude Code CLI** (no API key) - uses your existing Claude Code
subscription / session. Install from [claude.com/claude-code](https://claude.com/claude-code),
then run `claude` once and sign in. Verify with `claude --version`.
- **Codex CLI** (no API key) - uses your existing Codex session. Verify with
`codex --version` and sign in with `codex login` if needed.

### Quick setup with pip

Expand Down Expand Up @@ -136,12 +141,18 @@ $ cp .env.example .env

**Environment variables**

| Variable | Values | Description |
| ---------------- | ------------------------------------------- | ---------------------------------------------------------------------- |
| `LLM_PROVIDER` | `ollama` or `gemini` | Chooses provider. Defaults to Ollama. |
| `DEFAULT_MODEL` | for example `gemma3:4b` or `gemini-2.5-pro` | Model name passed to the provider. |
| `GEMINI_API_KEY` | string | Required when `LLM_PROVIDER=gemini`. |
| `GITHUB_TOKEN` | optional | Inherits from your shell environment, improves GitHub API rate limits. |
| Variable | Values | Description |
| ----------------------------- | ----------------------------------------------------- | --------------------------------------------------------------------------- |
| `LLM_PROVIDER` | `ollama`, `gemini`, `claude_code`, or `codex` | Chooses the provider. CLI providers are forced when set, regardless of model. |
| `DEFAULT_MODEL` | e.g. `gemma3:4b`, `gemini-2.5-pro`, `claude-code`, or `codex-cli` | Model name passed to the provider. Use `claude-code` or `codex-cli` for CLI mode. |
| `CLAUDE_CODE_COMMAND` | optional, default `claude` | Command or full path to the Claude Code CLI. |
| `CLAUDE_CODE_TIMEOUT_SECONDS` | optional, default `300` | Per-call timeout for the Claude Code CLI. |
| `CLAUDE_CODE_MODEL` | optional, e.g. `sonnet` | Pin a specific model for nested calls. Empty = your session default. |
| `CODEX_COMMAND` | optional, default `codex` | Command or full path to the Codex CLI. |
| `CODEX_TIMEOUT_SECONDS` | optional, default `300` | Per-call timeout for the Codex CLI. |
| `CODEX_MODEL` | optional | Pin a specific Codex model for nested calls. Empty = Codex CLI default. |
| `GEMINI_API_KEY` | string | Required when `LLM_PROVIDER=gemini`. |
| `GITHUB_TOKEN` | optional | Inherits from your shell environment, improves GitHub API rate limits. |

Provider mapping lives in `prompt.py` and `models.py`. The `config.py` file has a single flag:

Expand Down Expand Up @@ -209,6 +220,61 @@ Provide a path to a resume PDF.
$ python score.py /path/to/resume.pdf
```

> On systems where `python` is not on `PATH`, use `python3` instead:
>
> ```bash
> $ python3 score.py /path/to/resume.pdf
> ```

#### Run with Claude Code (no API key)

Set these values in `.env`:

```bash
LLM_PROVIDER=claude_code
DEFAULT_MODEL=claude-code
CLAUDE_CODE_COMMAND=claude
CLAUDE_CODE_TIMEOUT_SECONDS=300
```

Then run:

```bash
$ claude --version # confirm the CLI is installed and signed in
$ python3 score.py /path/to/resume.pdf
```

Each pipeline run makes **multiple** LLM calls per resume (one per resume
section, GitHub project selection, and the final evaluation), and every call
shells out to `claude`. Expect it to be **slower** than Ollama and to consume
Claude Code usage accordingly. Pin a cheaper model with `CLAUDE_CODE_MODEL`
(for example `sonnet`) to reduce cost.

#### Run with Codex (no API key)

Set these values in `.env`:

```bash
LLM_PROVIDER=codex
DEFAULT_MODEL=codex-cli
CODEX_COMMAND=codex
CODEX_TIMEOUT_SECONDS=300
# CODEX_MODEL=
```

Then run:

```bash
$ codex --version # confirm the CLI is installed
$ codex login # sign in if needed
$ python3 score.py /path/to/resume.pdf
```

Codex mode uses the same shared CLI provider wrapper as Claude Code mode, but
builds Codex-specific commands with `codex exec`, `--ephemeral`, read-only
sandboxing, and `--output-schema` for structured calls. Leave `CODEX_MODEL`
unset to use the Codex CLI's configured default model.

What happens:

1. If development mode is on, the PDF extraction result is cached to `cache/resumecache_<basename>.json`.
Expand Down Expand Up @@ -253,6 +319,35 @@ What happens:

## Provider details

### Claude Code (no API key)

- Set `LLM_PROVIDER=claude_code` and `DEFAULT_MODEL=claude-code`
- Requires the `claude` CLI installed and authenticated; no `ANTHROPIC_API_KEY`
or any other key is used
- `models.CliLLMProvider` shells out to `claude -p` in non-interactive print
mode with all built-in tools disabled (`--tools ""`) and project config
isolated (`--safe-mode`), so the nested Claude only answers the prompt; it
never edits files, runs commands, or acts as a recursive coding agent
- System messages are passed through `--system-prompt`; JSON schemas
(`kwargs["format"]`) are passed through Claude Code's native `--json-schema`
flag; the CLI's stdout is adapted to the unified
`{"message": {"content": ...}}` shape
- Tune with `CLAUDE_CODE_COMMAND`, `CLAUDE_CODE_TIMEOUT_SECONDS`, and the
optional `CLAUDE_CODE_MODEL`

### Codex CLI (no API key)

- Set `LLM_PROVIDER=codex` and `DEFAULT_MODEL=codex-cli`
- Requires the `codex` CLI installed and authenticated with `codex login`; no
separate API key is required
- `models.CliLLMProvider` shells out to `codex exec` with `--ephemeral`,
`--ignore-rules`, `--ignore-user-config`, and read-only sandboxing, so each
nested call behaves like a constrained text-generation step
- JSON schemas (`kwargs["format"]`) are written to a temporary schema file and
passed through Codex's native `--output-schema` flag
- Tune with `CODEX_COMMAND`, `CODEX_TIMEOUT_SECONDS`, and the optional
`CODEX_MODEL`

### Ollama

- Set `LLM_PROVIDER=ollama`
Expand Down
2 changes: 1 addition & 1 deletion github.py
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,7 @@ def generate_projects_json(projects: List[Dict]) -> List[Dict]:
)

print(
f"🤖 Using LLM to select top 5 projects from {len(projects)} repositories..."
f"🤖 Using LLM to select top 7 projects from {len(projects)} repositories..."
)

# Initialize the LLM provider
Expand Down
89 changes: 74 additions & 15 deletions llm_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,23 @@

import logging
from typing import Any, Dict, Optional
from models import ModelProvider, OllamaProvider, GeminiProvider
from prompt import MODEL_PROVIDER_MAPPING, GEMINI_API_KEY
from models import (
ModelProvider,
OllamaProvider,
GeminiProvider,
CliLLMProvider,
)
from prompt import (
MODEL_PROVIDER_MAPPING,
GEMINI_API_KEY,
PROVIDER,
CLAUDE_CODE_COMMAND,
CLAUDE_CODE_MODEL,
CLAUDE_CODE_TIMEOUT_SECONDS,
CODEX_COMMAND,
CODEX_MODEL,
CODEX_TIMEOUT_SECONDS,
)

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -37,26 +52,70 @@ def extract_json_from_response(response_text: str) -> str:
return response_text


CLI_PROVIDER_SETTINGS = {
ModelProvider.CLAUDE_CODE.value: {
"command": CLAUDE_CODE_COMMAND,
"timeout": CLAUDE_CODE_TIMEOUT_SECONDS,
"model": CLAUDE_CODE_MODEL,
"label": "Claude Code CLI",
},
ModelProvider.CODEX.value: {
"command": CODEX_COMMAND,
"timeout": CODEX_TIMEOUT_SECONDS,
"model": CODEX_MODEL,
"label": "Codex CLI",
},
}


def _build_cli_provider(provider_name: str) -> Any:
"""Construct a shared CLI provider from environment configuration."""
settings = CLI_PROVIDER_SETTINGS[provider_name]
logger.info(f"Using {settings['label']} provider (local authenticated session)")
return CliLLMProvider(
backend=provider_name,
command=settings["command"],
timeout=settings["timeout"],
model=settings["model"],
)


def initialize_llm_provider(model_name: str) -> Any:
"""
Initialize the appropriate LLM provider based on the model name.
Initialize the appropriate LLM provider.

Selection order:
1. An explicit CLI provider (``LLM_PROVIDER=claude_code`` or
``LLM_PROVIDER=codex``) wins regardless of the model name. (Previously
provider selection was driven solely by the model mapping, so
LLM_PROVIDER was ignored.)
2. Otherwise the provider is inferred from ``MODEL_PROVIDER_MAPPING``:
Gemini (requires ``GEMINI_API_KEY``, else falls back to Ollama),
Claude Code, or Ollama (the default fallback).

Args:
model_name: The name of the model to use
model_name: The name of the model to use.

Returns:
An initialized LLM provider (either OllamaProvider or GeminiProvider)
An initialized LLM provider implementing the ``chat`` interface.
"""
# Default to Ollama provider
provider = OllamaProvider()
# If using Gemini and API key is available, use Gemini provider
# 1. Explicit provider selection wins; this is what makes CLI-backed
# providers take effect even when DEFAULT_MODEL is just a logical label.
if PROVIDER in CLI_PROVIDER_SETTINGS:
return _build_cli_provider(PROVIDER)

# 2. Infer the provider from the model mapping (legacy behavior).
model_provider = MODEL_PROVIDER_MAPPING.get(model_name, ModelProvider.OLLAMA)

if model_provider.value in CLI_PROVIDER_SETTINGS:
return _build_cli_provider(model_provider.value)

if model_provider == ModelProvider.GEMINI:
if not GEMINI_API_KEY:
logger.warning("⚠️ Gemini API key not found. Falling back to Ollama.")
else:
logger.info(f"🔄 Using Google Gemini API provider with model {model_name}")
provider = GeminiProvider(api_key=GEMINI_API_KEY)
else:
logger.info(f"🔄 Using Ollama provider with model {model_name}")
return provider
logger.warning("Gemini API key not found. Falling back to Ollama.")
return OllamaProvider()
logger.info(f"Using Google Gemini API provider with model {model_name}")
return GeminiProvider(api_key=GEMINI_API_KEY)

logger.info(f"Using Ollama provider with model {model_name}")
return OllamaProvider()
Loading