Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable/Disable local LLM providers #175

Open
stephanj opened this issue Jul 21, 2024 · 0 comments
Open

Enable/Disable local LLM providers #175

stephanj opened this issue Jul 21, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@stephanj
Copy link
Contributor

stephanj commented Jul 21, 2024

Would be nice to disable certain local LLM providers, this way only the preferred one is shown by default.

We need to modify a few components of the plugin to make this happen

  1. First, let's add a new setting to the DevoxxGenieStateService class to store the enabled/disabled state of local providers:
@Getter
@Setter
@State(
    name = "com.devoxx.genie.ui.SettingsState",
    storages = @Storage("DevoxxGenieSettingsPlugin.xml")
)
public final class DevoxxGenieStateService implements PersistentStateComponent<DevoxxGenieStateService> {
    // ... existing code ...

    private Map<ModelProvider, Boolean> localProviderEnabledStates = new HashMap<>();

    // ... existing code ...

    public DevoxxGenieStateService() {
        // Initialize default states for local providers
        localProviderEnabledStates.put(ModelProvider.Ollama, true);
        localProviderEnabledStates.put(ModelProvider.LMStudio, true);
        localProviderEnabledStates.put(ModelProvider.GPT4All, true);
        localProviderEnabledStates.put(ModelProvider.Jan, true);
    }

    public boolean isLocalProviderEnabled(ModelProvider provider) {
        return localProviderEnabledStates.getOrDefault(provider, false);
    }

    public void setLocalProviderEnabled(ModelProvider provider, boolean enabled) {
        localProviderEnabledStates.put(provider, enabled);
    }
}
  1. Next, update the LLMProvidersComponent to include checkboxes for enabling/disabling local providers:
public class LLMProvidersComponent extends AbstractSettingsComponent {
    // ... existing code ...

    @Getter
    private final Map<ModelProvider, JCheckBox> localProviderCheckboxes = new HashMap<>();

    public LLMProvidersComponent() {
        // ... existing code ...

        initializeLocalProviderCheckboxes();
    }

    private void initializeLocalProviderCheckboxes() {
        DevoxxGenieStateService stateService = DevoxxGenieStateService.getInstance();
        for (ModelProvider provider : Arrays.asList(ModelProvider.Ollama, ModelProvider.LMStudio, ModelProvider.GPT4All, ModelProvider.Jan)) {
            JCheckBox checkbox = new JCheckBox(provider.getName(), stateService.isLocalProviderEnabled(provider));
            localProviderCheckboxes.put(provider, checkbox);
        }
    }

    @Override
    public JPanel createPanel() {
        // ... existing code ...

        addSection(panel, gbc, "Local Large Language Models");
        for (Map.Entry<ModelProvider, JCheckBox> entry : localProviderCheckboxes.entrySet()) {
            addSettingRow(panel, gbc, "Enable " + entry.getKey().getName(), entry.getValue());
        }

        // ... rest of the existing code ...
    }
}
  1. Update the LLMProvidersConfigurable to apply and reset the local provider settings:
public class LLMProvidersConfigurable implements Configurable {
    // ... existing code ...

    @Override
    public boolean isModified() {
        DevoxxGenieStateService settings = DevoxxGenieStateService.getInstance();
        
        // ... existing checks ...

        for (Map.Entry<ModelProvider, JCheckBox> entry : llmSettingsComponent.getLocalProviderCheckboxes().entrySet()) {
            if (settings.isLocalProviderEnabled(entry.getKey()) != entry.getValue().isSelected()) {
                return true;
            }
        }

        return false;
    }

    @Override
    public void apply() {
        DevoxxGenieStateService settings = DevoxxGenieStateService.getInstance();
        
        // ... existing apply logic ...

        for (Map.Entry<ModelProvider, JCheckBox> entry : llmSettingsComponent.getLocalProviderCheckboxes().entrySet()) {
            settings.setLocalProviderEnabled(entry.getKey(), entry.getValue().isSelected());
        }

        // Notify listeners that settings have changed
        ApplicationManager.getApplication().getMessageBus()
            .syncPublisher(AppTopics.SETTINGS_CHANGED_TOPIC)
            .settingsChanged(true);
    }

    @Override
    public void reset() {
        // ... existing reset logic ...

        DevoxxGenieStateService settings = DevoxxGenieStateService.getInstance();
        for (Map.Entry<ModelProvider, JCheckBox> entry : llmSettingsComponent.getLocalProviderCheckboxes().entrySet()) {
            entry.getValue().setSelected(settings.isLocalProviderEnabled(entry.getKey()));
        }
    }
}
  1. Modify the LLMProviderService to only return enabled local providers:
public class LLMProviderService {
    // ... existing code ...

    public List<ModelProvider> getLocalModelProviders() {
        DevoxxGenieStateService settings = DevoxxGenieStateService.getInstance();
        return Arrays.asList(ModelProvider.GPT4All, ModelProvider.LMStudio, ModelProvider.Ollama, ModelProvider.Jan)
            .stream()
            .filter(settings::isLocalProviderEnabled)
            .collect(Collectors.toList());
    }

    // ... rest of the existing code ...
}
  1. Finally, update the DevoxxGenieToolWindowContent class to refresh the model providers when settings change:
public class DevoxxGenieToolWindowContent implements SettingsChangeListener,
                                                     LLMSettingsChangeListener,
                                                     ConversationStarter,
                                                     CustomPromptChangeListener {
    // ... existing code ...

    @Override
    public void settingsChanged(boolean hasKey) {
        // ... existing code ...

        refreshModelProviders();
    }

    private void refreshModelProviders() {
        modelProviderComboBox.removeAllItems();
        addModelProvidersToComboBox();
        
        // Attempt to restore the previously selected provider if it's still available
        ModelProvider currentProvider = (ModelProvider) modelProviderComboBox.getSelectedItem();
        if (currentProvider == null && modelProviderComboBox.getItemCount() > 0) {
            modelProviderComboBox.setSelectedIndex(0);
        }

        updateModelNamesComboBox(currentProvider != null ? currentProvider.getName() : null);
    }

    // ... rest of the existing code ...
}

These changes will allow users to enable or disable local LLM providers through the settings UI. The enabled/disabled state will be persisted, and the combobox in the main UI will be updated accordingly. When a user changes these settings, the UI will refresh to show only the enabled providers.

Remember to update any other relevant parts of your code that might depend on the list of available providers, such as the ChatModelProvider class, to respect these new settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant