Skip to content

Add GGUF embedding-backbone export for grounding retrieval#1768

Open
maziyarpanahi wants to merge 1 commit into
masterfrom
feature/om-195-gguf-embedding-export
Open

Add GGUF embedding-backbone export for grounding retrieval#1768
maziyarpanahi wants to merge 1 commit into
masterfrom
feature/om-195-gguf-embedding-export

Conversation

@maziyarpanahi

Copy link
Copy Markdown
Owner

Description

Adds local F16 and Q8_0 GGUF export for embedding backbones used by the grounding retrieval path. The converter validates checkpoint configuration before starting, rejects token-classification heads with a descriptive error, writes both variants atomically, and emits publication-ready artifact metadata.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Code refactoring
  • Performance improvement
  • Test addition/improvement

Changes Made

  • Add an embedding-only GGUF converter with F16 and Q8_0 outputs.
  • Reject token-classification and other task-specific heads before conversion.
  • Record canonical GGUF format plus per-artifact precision and quantization metadata.
  • Document local setup, output layout, limitations, and manifest fields.

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this change with different models/inputs

Validation:

  • make format
  • make lint
  • make format-check
  • make docs-build
  • .venv/bin/python -m pytest tests/ -q (5,202 passed, 46 skipped)

Documentation

  • I have updated the documentation accordingly
  • I have added docstrings to new functions/classes
  • I have updated the CHANGELOG.md

Code Quality

  • I ran make format, make lint, and make format-check
  • For Swift/OpenMedKit changes, I ran make format-swift and make lint-swift
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings

Dependencies

  • I have not added any new dependencies
  • OR I have added new dependencies and they are justified because: ____

Checklist

  • I have read the contributing guidelines
  • My commits have clear, descriptive messages
  • I have squashed/organized my commits appropriately

Related Issues

Closes #1463

Screenshots/Examples

The documented output contains model-f16.gguf, model-q8_0.gguf, config.json, and openmed-gguf.json.

@maziyarpanahi maziyarpanahi added roadmap-v2 OpenMed V2 roadmap backlog feature New capability P3 Strategic labels Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New capability P3 Strategic roadmap-v2 OpenMed V2 roadmap backlog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add GGUF embedding-backbone export for the grounding retrieval path

1 participant