feat: dynamic Ollama catalog (v0.1.3)

AGENT.md

@@ -20,8 +20,7 @@ before writing any code.
 - **First runtime:** Ollama (shipped). Future: LM Studio, llama.cpp, Jan AI,
   GPT4All, vLLM (added as adapters, no core changes)
 - **Design style:** Clean Architecture + SOLID + Dependency Inversion
-- **Status:** Greenfield. No source code yet — only docs (`PROJECT.MD`,
-  `Architecture.md`, `README.md`, `LICENSE`).
+- **Status:** Active development. Dynamic catalog from ollama.com.
 
 ---
 
@@ -137,7 +136,6 @@ Domain:      modeldock/domain/ (pure entities, no I/O)
 Ports:       modeldock/ports/  (typing.Protocol interfaces)
 Adapters:    modeldock/adapters/ (runtimes, registry, downloaders, cache, progress)
 Common:      modeldock/common/ (config, logging, platform, http, errors)
-Data:        modeldock/data/catalog.json (bundled model registry)
 ```
 
 See `Architecture.md` §2 for the full tree and §3 for per-module responsibilities.

Architecture.md

@@ -48,7 +48,7 @@ installation verification, and loading through pluggable runtime adapters.
 │  Adapter / Infrastructure Layer (modeldock/adapters)         │
 │  - runtimes/ollama.py (first), lmstudio, llamacpp, jan,      │
 │    gpt4all, vllm (future)                                     │
-│  - registry/ (bundled catalog + remote registry)             │
+│  - registry/ (dynamic Ollama catalog + bundled fallback)      │
 │  - downloaders/ (http, ollama-native pull)                   │
 │  - cache/ (filesystem cache)                                 │
 │  - progress/ (rich, tqdm, silent)                            │
@@ -112,8 +112,9 @@ modeldock/
 │       │   │   ├── gpt4all.py
 │       │   │   └── vllm.py
 │       │   ├── registry/
-│       │   │   ├── bundled.py     # static catalog shipped in package
-│       │   │   └── remote.py      # optional remote registry fetch
+│       │   │   ├── ollama_library.py  # dynamic catalog from ollama.com (default)
+│       │   │   ├── bundled.py        # static fallback catalog (offline)
+│       │   │   └── remote.py         # optional remote registry fetch
 │       │   ├── downloaders/
 │       │   │   ├── http.py        # generic HTTP downloader
 │       │   │   └── ollama_pull.py # delegates to `ollama pull`/API
@@ -165,13 +166,13 @@ standard.
 | `ports/` | `typing.Protocol` interfaces defining *what* the system needs from the outside world (runtime, registry, downloader, cache, progress, events). No implementation. |
 | `core/` | Application services that implement use cases by composing ports. `LifecycleOrchestrator` is the brain behind `load()`. `ModelManager` is the high-level facade. |
 | `adapters/runtimes/` | Concrete runtime integrations. Each implements `RuntimePort`. Ollama ships first; others are stubs implementing the interface. |
-| `adapters/registry/` | Provides the searchable model catalog. `bundled.py` reads `catalog.json`; `remote.py` can refresh from a URL. |
+| `adapters/registry/` | Provides the searchable model catalog. `ollama_library.py` scrapes ollama.com for the full model list with auto-detected categories/capabilities; `bundled.py` is a static fallback; `remote.py` can refresh from a URL. |
 | `adapters/downloaders/` | Moves bytes. `ollama_pull.py` wraps Ollama's native pull; `http.py` is a generic fallback for runtimes that expose direct URLs. |
 | `adapters/cache/` | Tracks what is installed/downloaded so we never re-fetch. Filesystem manifest + content hashing. |
 | `adapters/progress/` | Pluggable progress reporters (rich, tqdm, silent) implementing `ProgressPort`. |
 | `cli/` | Thin delivery layer. Translates argv → core service calls. No business logic. |
 | `common/` | Config loading, logging bootstrap, platform/OS helpers, shared HTTP client, base errors. |
-| `data/catalog.json` | The bundled, versioned model registry (names, aliases, categories, capabilities, sizes, default tags). |
+| `data/catalog.json` | **Deprecated.** Static bundled model registry. Replaced by `OllamaLibraryRegistry` which scrapes ollama.com dynamically. Kept as offline fallback only. |
 
 ---
 
@@ -293,8 +294,8 @@ modeldock --help
   explicit runtime overrides.
 - **Format:** TOML for the file (human-friendly, stdlib `tomllib` in 3.11+).
 - **Model:** a frozen `Settings` dataclass/pydantic model: `default_backend`,
-  `cache_dir`, `registry_url`, `log_level`, `progress_style`, `auto_install`,
-  `ollama_host`, etc.
+  `cache_dir`, `registry_url`, `catalog_source`, `log_level`, `progress_style`,
+  `auto_install`, `ollama_host`, etc.
 - **Cross-platform paths:** resolved via `common/platform.py` using
   `platformdirs` (the de-facto standard for user/config/cache dirs across OSes).
 - **Validation:** config loaded through a validator; unknown keys warn, invalid
@@ -330,21 +331,35 @@ Goal: "never re-download installed models" + offline cache management.
 
 A **searchable, versioned catalog** decoupled from any runtime.
 
-- **Bundled catalog** (`data/catalog.json`): curated entries with
+- **Dynamic catalog** (`adapters/registry/ollama_library.py`): scrapes
+  `ollama.com/library` for the full model list. Auto-detects `Category` and
+  `Capability` from model names and HTML tags. Caches results locally
+  (`<cache_dir>/catalog_cache.json`) with 24-hour TTL for offline use.
+  This is the **default** registry.
+- **Bundled fallback** (`data/catalog.json`): static curated entries with
   `name, aliases[], category, capabilities[], default_tag,
   variants[{tag, params, size_bytes, min_ram}], description, backend_hints`.
-  Shipped with the package → works offline, zero-config.
+  Used as offline fallback when `catalog_source="bundled"` or when the
+  dynamic catalog fails and no cache exists.
 - **`RegistryPort`** abstraction: `search(query)`, `get(ref)`,
   `by_category(cat)`, `recommend(task)`, `list_all()`.
 - **Implementations:**
-  - `BundledRegistry` — reads `catalog.json` (default).
+  - `OllamaLibraryRegistry` — scrapes ollama.com, auto-detects metadata,
+    caches locally (default).
+  - `BundledRegistry` — reads `catalog.json` (offline fallback).
   - `RemoteRegistry` — optional fetch/refresh from a URL/JSON for community
     updates without a package release.
+- **Configuration:** `catalog_source` setting in `Settings` controls which
+  registry is used: `"auto"` (try dynamic, fallback to bundled), `"ollama"`
+  (dynamic only), `"bundled"` (static only).
 - **Alias resolution:** `domain/alias.py` maps friendly names (`llama3`) →
   canonical `ModelSpec`. Handles version shortcuts (`llama3:8b`),
   capability-based lookup (`recommend("vision")`), and "auto model selection."
-- **Extensibility:** new models added by editing `catalog.json` (or a remote
-  registry) — no code change. Future: community-contributed registry plugins.
+- **Auto-detection rules:**
+  - **Category:** name contains `embed` → EMBEDDING; `code` → CODING;
+    `vision`/`llava` → VISION; `r1`/`thinking` → REASONING; default → CHAT.
+  - **Capabilities:** HTML pills (`tools`, `vision`, `thinking`, `audio`,
+    `embedding`) map to `Capability` enum values. Default: CHAT + COMPLETION.
 
 ---
 
@@ -504,7 +519,7 @@ dependency-light. Runtime-native SDKs (e.g., `ollama` Python client) are
 |---|---|---|
 | Clean Architecture + ports | Maximum extensibility for 6 future runtimes; testable without Ollama | More files/abstraction upfront; slight "ceremony" for a small v1 |
 | Return runtime-native client from `load()` | Stay lightweight; don't reimplement inference | Less uniform cross-runtime inference API (acceptable — management is the product) |
-| Bundled `catalog.json` | Works offline, zero-config, beginner-friendly | Catalog can go stale; mitigated by optional `RemoteRegistry` refresh |
+| Dynamic catalog via HTML scraping | Always up-to-date with ollama.com; zero manual maintenance | Depends on ollama.com HTML structure; mitigated by 24h cache + bundled fallback |
 | Entry-point plugins | Third parties extend without forking | Slightly more discovery code; stdlib `importlib.metadata` is cheap |
 | `httpx` over `requests` | Streaming/resumable downloads, async-ready | Extra dep (small, well-maintained) |
 | Optional runtime SDK extras | Tiny base install | User may need `pip install modeldock[ollama]` (documented) |
@@ -516,7 +531,8 @@ dependency-light. Runtime-native SDKs (e.g., `ollama` Python client) are
 ## 17. Future Scalability Considerations
 
 - **New runtimes:** entry-point adapters — zero core changes (§14).
-- **New models:** catalog edits / remote registry — no code change.
+- **New models:** dynamic catalog scrapes ollama.com automatically; no manual
+  catalog edits needed. Models appear as soon as they're added to ollama.com.
 - **Concurrency:** `install_category` parallel downloads; service layer already
   async-ready (`httpx` async). Could add `asyncio` variants of the API
   (`md.aload(...)`) later without breaking sync API.

CHANGELOG.md

@@ -4,6 +4,43 @@ All notable changes to ModelDock will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/), and this project adheres to [Semantic Versioning](https://semver.org/).
 
+## [0.1.3] - 2026-07-19
+
+Dynamic catalog: replaced static `catalog.json` with live scraping of ollama.com.
+
+### Added
+
+- `OllamaLibraryRegistry` adapter — scrapes `ollama.com/library` for the full model list, auto-detects categories and capabilities, and caches locally for offline use
+- `catalog_source` config setting (`"auto"` | `"ollama"` | `"bundled"`) to control which registry is used
+- `MODELDOCK_CATALOG_SOURCE` environment variable support
+- Local catalog cache (`<cache_dir>/catalog_cache.json`) with 24-hour TTL
+
+### Changed
+
+- `ModelManager` now defaults to `OllamaLibraryRegistry` (dynamic) instead of `BundledRegistry` (static)
+- Auto-detection rules: model name patterns and HTML capability tags determine `Category` and `Capability`
+- `Architecture.md` updated to reflect dynamic catalog design
+
+### Removed
+
+- Deleted `src/modeldock/data/catalog.json` — no longer needed
+- Removed `[tool.setuptools.package-data]` from `pyproject.toml`
+
+### Deprecated
+
+- `BundledRegistry` is now a fallback only, used when `catalog_source="bundled"` or when the dynamic catalog fails and no cache exists
+
+### Tests
+
+- 32 new tests for `OllamaLibraryRegistry` (HTML scraping, auto-detection, cache, network fallback)
+- BundledRegistry tests skipped when `catalog.json` is not present
+
+### Contributors
+
+- @himanshu231204
+
+---
+
 ## [0.1.2] - 2026-07-19
 
 Patch fix for `catalog.json` not being included in the installed package.
@@ -81,6 +118,7 @@ This project follows [Semantic Versioning](https://semver.org/):
 
 ## Links
 
+[0.1.3]: https://github.com/OpenAgentHQ/modeldock/compare/v0.1.2...v0.1.3
 [0.1.2]: https://github.com/OpenAgentHQ/modeldock/compare/v0.1.1...v0.1.2
 [0.1.1]: https://github.com/OpenAgentHQ/modeldock/compare/v0.1.0...v0.1.1
 [0.1.0]: https://github.com/OpenAgentHQ/modeldock/releases/tag/v0.1.0

CONTEXT.md

@@ -14,7 +14,7 @@ inference itself; it orchestrates runtimes (starting with Ollama).
 
 - **Repo:** https://github.com/OpenAgentHQ/modeldock.git
 - **Language:** Python 3.9–3.12 · `src/` layout · pure-Python, minimal deps
-- **Status:** Greenfield (docs only; no source yet)
+- **Status:** Active development. Dynamic catalog from ollama.com.
 
 ---
 
@@ -41,7 +41,7 @@ inference itself; it orchestrates runtimes (starting with Ollama).
 3. **Extensible by design.** New runtimes = one adapter class + one entry-point
    line. No core/CLI/API changes.
 4. **Zero-config, beginner-friendly, cross-platform.** `platformdirs` for
-   paths; works offline via bundled `catalog.json`.
+   paths; works offline via cached catalog from ollama.com.
 5. **Typed errors, library-friendly logging.** Never `basicConfig()` at import;
    raise `ModelDockError` subclasses with actionable context.
 
@@ -56,7 +56,6 @@ Domain:      modeldock/domain/ (pure entities, no I/O)
 Ports:       modeldock/ports/  (typing.Protocol interfaces)
 Adapters:    modeldock/adapters/ (runtimes, registry, downloaders, cache, progress)
 Common:      modeldock/common/ (config, logging, platform, http, errors)
-Data:        modeldock/data/catalog.json (bundled model registry)
 ```
 
 **Key ports:** `RuntimePort`, `RegistryPort`, `DownloaderPort`, `CachePort`,

Development.md

@@ -158,7 +158,7 @@ python -m modeldock --help
 
 ## 8. Configuration During Development
 
-Config precedence (low → high): built-in defaults → bundled `catalog.json` →
+Config precedence (low → high): built-in defaults → dynamic catalog (ollama.com) →
 system config → user config (`~/.config/modeldock/config.toml` or
 `%APPDATA%\modeldock\config.toml`) → env vars (`MODELDOCK_*`) → runtime overrides.
 
@@ -200,7 +200,11 @@ See `AGENT.md` → "Extension Checklist" for the full list.
 
 ## 10. Adding Models to the Registry
 
-Edit `src/modeldock/data/catalog.json` (no code change). Each entry:
+Model discovery is **dynamic** — scraped live from `ollama.com/library`.
+New models appear automatically after catalog refresh (24h TTL cache).
+
+To add a model to the **bundled fallback**, edit `src/modeldock/data/catalog.json`
+(no code change). Each entry:
 
 ```json
 {

INSTRUCTIONS.md

@@ -93,7 +93,7 @@ src/modeldock/
   adapters/    runtimes, registry, downloaders, cache, progress
   cli/         Typer CLI, thin wrappers over core
   common/      config, logging, platform, http, errors
-  data/        catalog.json (bundled registry)
+  data/        (removed — catalog is now dynamic from ollama.com)
 ```
 
 **Dependency direction (never violate):**
@@ -147,7 +147,11 @@ Targets: ruff clean, `mypy --strict` clean, bandit clean, ≥80% coverage.
 
 ## 7. Adding Models
 
-Edit `src/modeldock/data/catalog.json`. No code change. See
+Model discovery is now **dynamic** — scraped live from `ollama.com/library`.
+New models appear automatically after catalog refresh (24h TTL cache).
+
+To add a model to the **bundled fallback**, edit `src/modeldock/data/catalog.json`
+(temporarily restored for offline fallback only). See
 [`Development.md`](./Development.md) §10 for the entry shape.
 
 ---

PROJECT.MD

@@ -294,6 +294,7 @@ model = load("llama3")
 
 ModelDock automatically:
 
+* **Scrapes a live catalog** from `ollama.com/library` (cached 24h, falls back to bundled)
 * Checks whether the model is already installed
 * Downloads the model if it is missing
 * Displays download progress

QUICKSTART.md

@@ -115,6 +115,17 @@ modeldock config show
 modeldock config set auto_install true
 ```
 
+### Catalog Source
+
+ModelDock scrapes `ollama.com/library` for a live model catalog, cached locally
+for 24 hours. Set `catalog_source` in config or `MODELDOCK_CATALOG_SOURCE` env var:
+
+| Value | Behavior |
+|-------|----------|
+| `auto` | Try dynamic, fallback to bundled (default) |
+| `ollama` | Dynamic only — requires internet |
+| `bundled` | Static catalog.json only — fully offline |
+
 ---
 
 ## Supported Runtimes

README.md

@@ -38,7 +38,7 @@ write `md.load("llama3")` and ModelDock handles the rest.
 - **Smart caching** — never re-download installed models; content-addressed offline cache.
 - **Extensible runtimes** — Ollama ships first; LM Studio, llama.cpp, Jan AI, GPT4All, vLLM are drop-in adapters.
 - **Cross-platform** — Windows, macOS, Linux via `platformdirs`.
-- **Zero-config, beginner-friendly** — works offline with a bundled catalog.
+- **Zero-config, beginner-friendly** — dynamic catalog from ollama.com with offline caching.
 
 ## Quick Start
 
@@ -128,9 +128,19 @@ Domain:      modeldock/domain/ (pure entities, no I/O)
 Ports:       modeldock/ports/  (typing.Protocol interfaces)
 Adapters:    modeldock/adapters/ (runtimes, registry, downloaders, cache, progress)
 Common:      modeldock/common/ (config, logging, platform, http, errors)
-Data:        modeldock/data/catalog.json (bundled model registry)
 ```
 
+### Catalog Source
+
+ModelDock scrapes `ollama.com/library` for a live model catalog, cached locally
+for 24 hours. Set `catalog_source` in config or `MODELDOCK_CATALOG_SOURCE` env var:
+
+| Value | Behavior |
+|-------|----------|
+| `auto` | Try dynamic, fallback to bundled (default) |
+| `ollama` | Dynamic only — requires internet |
+| `bundled` | Static catalog.json only — fully offline |
+
 See [Architecture.md](Architecture.md) for the full design contract.
 
 ## Configuration

SECURITY.md

@@ -45,7 +45,7 @@ Send an email to **opensource@openagenthq.com** with:
 - Keep ModelDock and its dependencies up to date (`pip install -U modeldock`).
 - Use environment variables for any secrets; never hardcode them.
 - Download models only from trusted runtimes/registries.
-- Review the bundled `catalog.json` and any remote registry sources.
+- Review the dynamic catalog source (`ollama.com`) and any bundled registry sources.
 
 ### For Contributors
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "modeldock"
-version = "0.1.2"
+version = "0.1.3"
 description = "Lightweight, Python-first model manager for local LLMs (the package manager for local AI models)."
 readme = "README.md"
 requires-python = ">=3.9"
@@ -56,9 +56,6 @@ ollama = "modeldock.adapters.runtimes.ollama:OllamaRuntime"
 [tool.setuptools.packages.find]
 where = ["src"]
 
-[tool.setuptools.package-data]
-modeldock = ["data/catalog.json"]
-
 [tool.ruff]
 line-length = 100
 target-version = "py39"

src/modeldock/__init__.py

@@ -8,7 +8,7 @@
 
 from typing import Any, Dict, List, Optional
 
-__version__ = "0.1.2"
+__version__ = "0.1.3"
 __author__ = "Himanshu kumar"
 
 from modeldock.common.config import Settings

src/modeldock/adapters/__init__.py

@@ -8,7 +8,7 @@
     TqdmProgress,
     make_progress,
 )
-from modeldock.adapters.registry import BundledRegistry, RemoteRegistry
+from modeldock.adapters.registry import BundledRegistry, OllamaLibraryRegistry, RemoteRegistry
 from modeldock.adapters.runtimes import (
     BaseRuntime,
     Gpt4AllRuntime,
@@ -30,6 +30,7 @@
     "VllmRuntime",
     "RuntimeRegistry",
     "BundledRegistry",
+    "OllamaLibraryRegistry",
     "RemoteRegistry",
     "HttpDownloader",
     "OllamaPullDownloader",

src/modeldock/adapters/registry/__init__.py

@@ -1,6 +1,7 @@
-"""ModelDock registry adapters — bundled catalog + optional remote."""
+"""ModelDock registry adapters — dynamic Ollama catalog + bundled fallback."""
 
 from modeldock.adapters.registry.bundled import BundledRegistry
+from modeldock.adapters.registry.ollama_library import OllamaLibraryRegistry
 from modeldock.adapters.registry.remote import RemoteRegistry
 
-__all__ = ["BundledRegistry", "RemoteRegistry"]
+__all__ = ["BundledRegistry", "OllamaLibraryRegistry", "RemoteRegistry"]