Pull Kluster Models Automatically

[themacexpert]

Kluster supports a variety of models including:

• deepseek-ai/DeepSeek-R1 — DeepSeek R1
• deepseek-ai/DeepSeek-V3-0324 — DeepSeek V3 (2025-03-24)
• google/gemma-3-27b-it — Gemma 3 27B
• klusterai/Meta-Llama-3.1-8B-Instruct-Turbo — Meta Llama 3.1 8B
• klusterai/Meta-Llama-3.3-70B-Instruct-Turbo — Meta Llama 3.3 70B
• meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 — Meta Llama 4 Maverick
• meta-llama/Llama-4-Scout-17B-16E-Instruct — Meta Llama 4 Scout
• mistralai/Mistral-Nemo-Instruct-2407 — Mistral NeMo
• Qwen/Qwen2.5-VL-7B-Instruct — Qwen 2.5 VL 7B
• Qwen/Qwen3-235B-A22B-FP8 — Qwen3-235B-A22B

This PR automatically fetches the supported models.
Let me know if I can help with any changes. Thanks!

[gemini-code-assist]

Hello @themacexpert, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello! Gemini here, providing a summary of this pull request to help everyone get up to speed quickly.

The primary goal of this PR, as indicated by the title and description, is to automate the process of fetching the list of available models from Kluster. Previously, the list of Kluster models was likely hardcoded or manually updated. This change introduces logic to call the Kluster API directly to retrieve the current list of supported models, ensuring the information is always up-to-date.

Highlights

• Automated Kluster Model Fetching: A new function fetch_kluster_models has been added to query the Kluster API (https://api.kluster.ai/v1/models) and retrieve the list of available models programmatically.
• Integration into Fetching Logic: The new fetch_kluster_models function has been integrated into the main script's execution flow, allowing it to be fetched concurrently with other providers (if enabled) or sequentially.
• Dynamic Output Generation: The script now uses the dynamically fetched Kluster models to generate the output markdown, replacing the previous static entry for Kluster.

Changelog

• src/pull_available_models.py
  • Reordered import statements for consistency (Diff 1).
  • Added the fetch_kluster_models function to fetch models from the Kluster API (Diff 2).
  • Added a logger specifically for Kluster (Diff 4).
  • Integrated fetch_kluster_models into the concurrent fetching logic using ThreadPoolExecutor (Diff 5, 6).
  • Integrated fetch_kluster_models into the sequential fetching logic (Diff 7).
  • Removed the static 'Kluster' entry from the trial_providers list (Diff 10).
  • Added logic to generate markdown output for Kluster using the dynamically fetched models (Diff 11).
  • Made minor quote style fixes in f-strings for Gemini and Llama models (Diff 3, 8, 9).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

An API call,
Models listed for all,
Kluster's list now true.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a valuable feature by dynamically fetching models from the Kluster API, which is a great step towards keeping the model list up-to-date automatically. The integration into the existing concurrent and sequential fetching logic, as well as the markdown generation, is well-handled.

I've identified one high-severity issue related to JSON parsing in the new fetch_kluster_models function that could lead to runtime errors. Addressing this will significantly improve the robustness of the new functionality. There are also a few minor points noted in the summary that could be considered for further refinement but are below the requested severity threshold for direct comments.

Summary of Findings

• JSON Parsing Robustness: The fetch_kluster_models function's JSON parsing logic could fail with an AttributeError if the API returns an unexpected dictionary structure (e.g., an error object instead of a model list or a dict containing a model list under the 'data' key) or if the model list contains non-dictionary items. This was commented on as a high-severity issue.
• Hardcoded API URL: The Kluster API URL (https://api.kluster.ai/v1/models) in fetch_kluster_models is hardcoded. Consider defining it as a module-level constant for better maintainability. (Severity: Low/Medium, not commented due to review settings)
• Unnecessary HTTP Header: The Content-Type: application/json header is included in the GET request to the Kluster API. This header is generally not needed for GET requests as they typically don't have a body. (Severity: Low, not commented due to review settings)
• Whitespace Issues (PEP 8): Several lines in the fetch_kluster_models function (e.g., lines 148, 151, 158) contain trailing whitespace or are empty lines with only whitespace. These should be removed or be truly blank lines for stylistic consistency. (Severity: Low, not commented due to review settings)

Merge Readiness

This pull request adds a useful feature for automatically fetching Kluster models. However, there is a high-severity issue concerning the robustness of JSON parsing that could lead to runtime errors. I recommend addressing this issue before merging to ensure the stability of the new functionality. As an AI, I am not authorized to approve pull requests; please ensure further review and approval from authorized team members after the suggested changes are made.

[gemini-code-assist]

The current JSON response handling for Kluster models might not be fully robust. If the API returns a dictionary that isn't the expected {'data': [...]} structure (e.g., an error object like {"error": "message"}) or if the list of models contains non-dictionary items, the code could raise an AttributeError when model.get('id') is called on an unexpected data type (like a string key from an error dictionary, or a non-dict item in the list).

Could we enhance the parsing logic to more strictly validate the structure before attempting to process the models? This would involve checking if api_response_data['data'] (if api_response_data is a dict) is indeed a list, and also ensuring that each item within the models_data list is a dictionary before trying to call .get() on it.

        # Parse the JSON response
        api_response_data = r.json()

        models_data = [] # Initialize as an empty list
        if isinstance(api_response_data, dict) and 'data' in api_response_data:
            if isinstance(api_response_data.get('data'), list):
                models_data = api_response_data['data']
            else:
                logger.error(f"Kluster API: 'data' field is present but not a list. Response: {{api_response_data}}")
                return [] # Return empty list on unexpected 'data' type
        elif isinstance(api_response_data, list):
            models_data = api_response_data
        else:
            logger.error(f"Unexpected JSON structure from Kluster API. Expected a list or a dict with a 'data' key. Response: {{api_response_data}}")
            return [] # Return empty list on wholly unexpected structure

        logger.info(f"Fetched {{len(models_data)}} models from Kluster")

        ret_models = []
        for model_obj in models_data:
            # Ensure each item is a dictionary before processing
            if not isinstance(model_obj, dict):
                logger.warning(f"Skipping non-dictionary item in Kluster models list: {{model_obj}}")
                continue

            # Extract fields from the model object
            model_id = model_obj.get('id')
            model_name = model_obj.get('name', model_id)

            # Skip models without an ID
            if not model_id:
                continue

            ret_models.append({
                "id": model_id,
                "name": model_name,  # Use actual name rather than lookup, as these are official names
            })

[cheahjs]

Thanks, merged in 3e1acfa