feat: filtering adapter

[thushan]

This PR brings in a generic filter adapter to configuration bits. It supports include and exclude types with glob support for now.

Enabled for profiles (so you can avoid say vllm profile when you don't have vllm) and models for an endpoint.

proxy:
  engine: "sherpa"
  load_balancer: "priority"
  # Only load profiles that support embeddings
  profile_filter:
    include:
      - "ollama"
      - "openai*"
    exclude:
      - "lm-studio"  # Doesn't have good embedding support

discovery:
  static:
    endpoints:
      - url: "http://localhost:11434"
        name: "embedding-server"
        type: "ollama"
        priority: 100
        model_filter:
          include:
            - "*embed*"      # Embedding models
            - "bge-*"        # BGE models
            - "e5-*"         # E5 models
            - "nomic-*"      # Nomic models
          exclude:
            - "*test*"       # No test models

This should help with the likes of LiteLLM etc and future work to enable #47 so you can exclude/include models as necessary (or even create model specific endpoints).

Summary by CodeRabbit

• New Features

  • Advanced filtering for profiles and models using glob include/exclude patterns.
  • Per-endpoint model filtering in discovery.
  • Runtime-configurable profile filtering in the loader.

• Documentation

  • New “Filters” page explaining configuration, precedence, and examples.
  • Reference updated with profile_filter and endpoint model_filter.
  • Added “Filtering Examples” section and navigation entry.

• Bug Fixes

  • Safer static discovery initialisation to prevent incomplete repository setup on failure.

• Tests

  • Extensive unit and integration tests for filtering behaviour, repository persistence, and discovery.

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

Introduces a glob-pattern filtering system for profiles and models, wiring it into profile loading and endpoint discovery. Adds domain types and ports, a GlobFilter implementation with repository, config fields, and utilities. Updates discovery and profile registry to use filters. Expands documentation and README. Adds extensive unit and integration tests.

Changes

Cohort / File(s)	Summary
Docs: Filtering guides and references docs/content/configuration/filters.md, docs/content/configuration/examples.md, docs/content/configuration/reference.md, docs/mkdocs.yml, readme.md	Adds new filtering docs, examples, and reference updates; updates navigation; README mentions advanced filtering. Note: examples.md includes the new section twice.
Domain and Ports: Filtering primitives internal/core/domain/filter.go, internal/core/ports/filter.go, internal/util/pattern/glob.go	Adds FilterConfig, FilterResult, FilterStats; introduces Filter and FilterRepository interfaces; provides case-insensitive glob matcher utility.
Filter adapter: Implementation and tests internal/adapter/filter/glob_filter.go, internal/adapter/filter/filter_repository.go, internal/adapter/filter/glob_filter_test.go, internal/adapter/filter/integration_test.go	Implements GlobFilter and in-memory FilterRepository; comprehensive unit and integration tests for matching, caching, maps/slices, validation, repo concurrency, and benchmarks.
Profile registry: Loader and factory changes internal/adapter/registry/profile/loader.go, internal/adapter/registry/profile/factory.go, internal/adapter/registry/profile/configurable_profile.go, test/integration/filter_profile_break_test.go	Makes ProfileLoader filter-aware with pluggable adapter and config; exposes Factory.GetLoader; switches capability matching to shared glob util; adds integration tests affecting profile availability and fallbacks.
Discovery: Endpoint model filtering internal/adapter/discovery/service.go, internal/adapter/discovery/repository.go, internal/adapter/discovery/endpoint_filter_test.go, internal/app/services/discovery.go	Adds per-endpoint model filtering with precedence and logging; propagates ModelFilter from config; new repo constructor with factory; safer static repo initialisation; extensive tests for preservation and override behaviour.
Config and Domain: Endpoint/Profile filter fields internal/config/types.go, internal/core/domain/endpoint.go	Adds YAML-backed ProfileFilter (proxy) and ModelFilter (endpoint); Endpoint includes ModelFilter; removes LoadFromConfig from EndpointRepository interface.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant App as Application
  participant Factory as Profile Factory
  participant Loader as ProfileLoader
  participant Filter as Filter (Glob)
  participant FS as Profiles Dir / Built-ins

  App->>Factory: NewProfileLoaderWithFilter(dir, profile_filter, glob)
  Factory->>Loader: construct with filter+config
  App->>Loader: LoadProfiles()
  Loader->>FS: Load built-ins + YAML
  FS-->>Loader: allProfiles map
  Loader->>Filter: ApplyToMap(profile_filter, allProfiles)
  alt no/empty filter
    Loader->>Loader: use allProfiles
  else filtered
    Filter-->>Loader: filteredProfiles map
    Loader->>Loader: assign filtered to internal registry
  end
  Loader-->>App: profiles ready

Loading

sequenceDiagram
  autonumber
  participant Boot as Startup
  participant Repo as StaticEndpointRepository
  participant Disc as ModelDiscoveryService
  participant E as Endpoint
  participant Client as DiscoveryClient
  participant Filter as Filter (Glob)
  participant Reg as ModelRegistry

  Boot->>Repo: LoadFromConfig(endpoints...)
  Repo-->>Boot: endpoints with ModelFilter
  Boot->>Disc: NewModelDiscoveryService(glob)
  loop each endpoint
    Disc->>E: getEndpointFilterConfig()
    Disc->>Client: DiscoverModels(E)
    Client-->>Disc: models[]
    alt filter present
      Disc->>Filter: Apply(filterConfig, models[])
      Filter-->>Disc: Accepted/Rejected
      Disc->>Reg: Register(Accepted)
    else no filter
      Disc->>Reg: Register(models[])
    end
  end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• feat: olla profile #32 — Also introduces FilterConfig, GlobFilter, profile-loader filtering, and discovery model_filter plumbing; overlaps heavily with this PR’s code paths.
• feat: API Redesign #33 — Modifies profile factory/loader behaviour and routing; touches the same registry components impacted here, potentially interacting with loader/factory changes.

Suggested labels

documentation

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

---

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 7f91ce3 and 140833f.

📒 Files selected for processing (22)

• docs/content/configuration/examples.md (1 hunks)
• docs/content/configuration/filters.md (1 hunks)
• docs/content/configuration/reference.md (3 hunks)
• docs/mkdocs.yml (1 hunks)
• internal/adapter/discovery/endpoint_filter_test.go (1 hunks)
• internal/adapter/discovery/repository.go (2 hunks)
• internal/adapter/discovery/service.go (5 hunks)
• internal/adapter/filter/filter_repository.go (1 hunks)
• internal/adapter/filter/glob_filter.go (1 hunks)
• internal/adapter/filter/glob_filter_test.go (1 hunks)
• internal/adapter/filter/integration_test.go (1 hunks)
• internal/adapter/registry/profile/configurable_profile.go (5 hunks)
• internal/adapter/registry/profile/factory.go (1 hunks)
• internal/adapter/registry/profile/loader.go (9 hunks)
• internal/app/services/discovery.go (1 hunks)
• internal/config/types.go (3 hunks)
• internal/core/domain/endpoint.go (1 hunks)
• internal/core/domain/filter.go (1 hunks)
• internal/core/ports/filter.go (1 hunks)
• internal/util/pattern/glob.go (1 hunks)
• readme.md (2 hunks)
• test/integration/filter_profile_break_test.go (1 hunks)

✨ Finishing Touches

• 📝 Generate Docstrings

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feature/filter-adapter

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

• Visit our Status Page to check the current availability of CodeRabbit.
• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.