Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
d7be26f
feat: Add VS Code Language Model API integration (#80)
ankit-apk Aug 15, 2025
21405f6
refactor: Address PR feedback for VS Code LM API integration
ankit-apk Aug 18, 2025
1fead53
cleanup: Remove unused files after merging changes
ankit-apk Aug 18, 2025
c63fe7f
fix: Register VS Code LM commands and fix compilation issues
ankit-apk Aug 18, 2025
a89a3e5
chore: Update .gitignore to exclude test artifacts
ankit-apk Aug 18, 2025
5409c08
docs: Update PR description with test results and review feedback
ankit-apk Aug 18, 2025
5e6e33a
refactor: address PR feedback - simplify implementation
ankit-apk Aug 18, 2025
1232c08
fix: update code review example to pass tests
ankit-apk Aug 18, 2025
58d3c14
refactor: address all PR review feedback
ankit-apk Aug 18, 2025
cb42de1
refactor: address all PR review feedback
ankit-apk Aug 18, 2025
8a4d65b
Merge branch 'feature/vscode-lm-api-integration' of https://github.co…
ankit-apk Aug 18, 2025
7bd1a65
Merge upstream/main - resolve conflicts in VS Code LM API integration
ankit-apk Aug 24, 2025
9d46562
merge upstream/main
ultmaster Sep 2, 2025
c6b019a
revert bump version
ultmaster Sep 2, 2025
fb71b55
refactor vscode command
ultmaster Sep 2, 2025
16741ef
Add comments
ultmaster Sep 2, 2025
fa49518
fix command bugs
ultmaster Sep 2, 2025
458e505
fix example
ultmaster Sep 2, 2025
6e05d27
restore tags and version
ultmaster Sep 2, 2025
b26db04
restore tags and version
ultmaster Sep 2, 2025
1ad65d8
Merge branch 'feature/vscode-lm-api-integration' of https://github.co…
ultmaster Sep 2, 2025
7097dbb
fix tests
ultmaster Sep 2, 2025
743b6e6
revert
ultmaster Sep 2, 2025
9cfdf9f
minor fix
ultmaster Sep 2, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,6 @@ poml-*.tgz
/mlartifacts
/site
/docs/typescript/reference/**/*.md
test-examples/
*.vsix
test-*.js
118 changes: 118 additions & 0 deletions PR_DESCRIPTION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Add VS Code Language Model API Integration

## Summary

This PR implements comprehensive support for VS Code's Language Model API, allowing POML users to leverage GitHub Copilot and other language models without configuring API keys. This addresses issue #80 and significantly improves the user experience.

## Changes Made (Updated After Review)

### Based on PR Feedback
- ✅ Merged `testCommandEnhanced.ts` directly into existing `testCommand.ts`
- ✅ Removed unnecessary `gallery/api_doc_generation.poml`
- ✅ Simplified code review example (removed complex stylesheet)
- ✅ Fixed all TypeScript compilation errors
- ✅ Properly registered commands in extension

### Core Implementation
1. **VS Code LM Provider** (`packages/poml-vscode/providers/vscodeLMProvider.ts`)
- Complete integration with VS Code Language Model API
- Automatic model detection and validation
- Streaming support with cancellation tokens

2. **Integrated Test Command** (`packages/poml-vscode/command/testCommand.ts`)
- Seamlessly integrated VS Code LM support
- Automatic fallback when no API key configured
- No duplicate command files

3. **New Commands** (`packages/poml-vscode/command/detectModelsCommand.ts`)
- `poml.detectVSCodeModels` - Detect available language models
- `poml.autoConfigureLM` - Auto-configure POML to use VS Code LM

## Testing Results ✅

Successfully tested with GitHub Copilot:
```
[info] Using VS Code Language Model API
[info] Testing prompt with chat model: /path/to/test.poml
Hello! Did you know that the first computer programmer was Ada Lovelace...
[info] Test completed in 6 seconds
```

### Test Coverage
- ✅ Command registration working
- ✅ Auto-configuration successful
- ✅ Model detection functional
- ✅ Response streaming working
- ✅ No API key required
- ✅ Fallback logic verified

## Features

### 🚀 Zero Configuration
- Automatically detects GitHub Copilot
- No API keys required
- One-click setup

### 🔄 Smart Fallback
- Uses VS Code LM when no API key configured
- Maintains backward compatibility
- Graceful error handling

### 🎯 User Experience
- "Using VS Code Language Model API" confirmation in output
- Clear error messages
- Seamless integration with existing workflows

## Usage

### Quick Setup
1. Install POML extension
2. Run: `POML: Auto-Configure Language Model`
3. Start using with GitHub Copilot!

### Manual Configuration
```json
{
"poml.languageModel.provider": "vscode",
"poml.languageModel.model": "copilot/gpt-4o"
}
```

## Compatibility

- Requires VS Code 1.95.0+
- Works with GitHub Copilot subscription
- Fully backward compatible with existing configurations

## Review Checklist

- [x] Code compiles without errors
- [x] Tests pass
- [x] Commands properly registered
- [x] Documentation updated
- [x] PR feedback addressed
- [x] Manually tested with GitHub Copilot
- [x] No duplicate files or unnecessary additions

## Screenshots

### Successful Test Output
```
2025-08-18 11:59:58.951 [info] Using VS Code Language Model API
2025-08-18 12:00:04.966 [info] Test completed in 6 seconds
```

### Auto-Configuration Success
"POML has been configured to use VS Code Language Model API. You can now test your prompts without configuring API keys."

## Related Issues

Fixes #80: "make the extension use VS Code LM API"

## Breaking Changes

None. Fully backward compatible.

## Acknowledgments

Thanks to @ultmaster for the helpful review feedback that led to a cleaner implementation.
222 changes: 222 additions & 0 deletions docs/vscode/vscode-lm-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
# Using VS Code Language Model API with POML

POML now supports VS Code's built-in Language Model API, allowing you to use GitHub Copilot and other language models without configuring API keys.

## Overview

The VS Code Language Model API integration enables POML to:
- Use GitHub Copilot's language models directly
- Automatically detect available models
- Work without API key configuration
- Seamlessly fall back to VS Code LM when no other provider is configured

## Requirements

- VS Code version 1.95.0 or later
- Active GitHub Copilot subscription (or other language model provider)
- POML extension installed

## Quick Start

### Automatic Configuration

1. Open any `.poml` file in VS Code
2. Run the command: `POML: Auto-Configure Language Model`
3. If GitHub Copilot is available, POML will automatically configure itself to use it

### Manual Configuration

1. Open VS Code Settings (`Cmd/Ctrl + ,`)
2. Search for "POML Language Model"
3. Set the following:
- **Provider**: `VS Code LM (GitHub Copilot)`
- **Model**: `copilot/gpt-4o` (or leave blank for auto-detection)
- **API Key**: Leave blank (not needed for VS Code LM)

### Detecting Available Models

To see which language models are available in your VS Code instance:

1. Run command: `POML: Detect VS Code Language Models`
2. The extension will show you all available models
3. Choose "Use VS Code LM" to automatically configure POML

## Supported Models

When using VS Code LM API, the following models are typically available:

- `copilot/gpt-4o` - GPT-4 Optimized (recommended)
- `copilot/gpt-4o-mini` - Smaller, faster variant
- `copilot/claude-3.5-sonnet` - Claude 3.5 Sonnet
- `copilot/o1` - OpenAI o1 model
- `copilot/o1-mini` - OpenAI o1 mini model

## Configuration Options

### settings.json

```json
{
"poml.languageModel.provider": "vscode",
"poml.languageModel.model": "copilot/gpt-4o",
"poml.languageModel.temperature": 0.7,
"poml.languageModel.maxTokens": 2000
}
```

### Provider Selection Priority

POML uses the following priority order for selecting a language model provider:

1. **Explicit VS Code LM**: If provider is set to "vscode"
2. **Automatic VS Code LM**: If no API key is configured and VS Code LM is available
3. **Configured Provider**: If API key and provider are configured

## Features

### Automatic Fallback

If you haven't configured any language model settings, POML will automatically try to use VS Code's Language Model API if available.

### User Consent

The first time you use VS Code LM API, you may be prompted to grant consent. This is a one-time authorization that allows extensions to use your language model subscription.

### Rate Limiting

VS Code LM API respects rate limits set by your language model provider. POML will handle rate limiting errors gracefully and provide appropriate feedback.

## Advantages

1. **No API Key Management**: Use your existing GitHub Copilot subscription
2. **Automatic Updates**: Models are updated automatically through VS Code
3. **Unified Billing**: Costs are included in your GitHub Copilot subscription
4. **Better Security**: No need to store API keys in settings
5. **Seamless Integration**: Works with VS Code's built-in authentication

## Troubleshooting

### "No language models found"

**Solution**:
- Ensure you're signed in to GitHub Copilot
- Run: `GitHub Copilot: Sign In` from the command palette
- Update VS Code to version 1.95.0 or later

### "User consent required"

**Solution**:
- This is normal for first-time use
- Click "Allow" when prompted
- The consent is remembered for future sessions

### "Rate limit exceeded"

**Solution**:
- Wait a few moments before trying again
- Check your GitHub Copilot usage limits
- Consider using a model with higher rate limits

### Models not appearing

**Solution**:
1. Check VS Code version: `Help > About`
2. Verify GitHub Copilot extension is installed and active
3. Run `POML: Detect VS Code Language Models` to refresh

## Migration Guide

### From OpenAI API

```json
// Before
{
"poml.languageModel.provider": "openai",
"poml.languageModel.model": "gpt-4",
"poml.languageModel.apiKey": "sk-..."
}

// After
{
"poml.languageModel.provider": "vscode",
"poml.languageModel.model": "copilot/gpt-4o"
// No API key needed!
}
```

### From Azure OpenAI

```json
// Before
{
"poml.languageModel.provider": "microsoft",
"poml.languageModel.model": "gpt-4-deployment",
"poml.languageModel.apiKey": "...",
"poml.languageModel.apiUrl": "https://....openai.azure.com"
}

// After
{
"poml.languageModel.provider": "vscode",
"poml.languageModel.model": "copilot/gpt-4o"
}
```

## Best Practices

1. **Model Selection**: Use `copilot/gpt-4o` for best performance and quality
2. **Temperature**: Adjust temperature based on your use case (0.0-1.0)
3. **Token Limits**: Be aware of model token limits (GPT-4o supports up to 64K tokens)
4. **Error Handling**: Implement fallback logic for when models are unavailable

## Example Usage

### Basic Prompt Testing

1. Create a `.poml` file:
```xml
<poml>
<role>You are a helpful assistant.</role>
<task>Explain quantum computing in simple terms.</task>
</poml>
```

2. Click the "Test" button or run `POML: Test current prompt on Chat Models`
3. POML will automatically use VS Code LM if configured

### Programmatic Usage

```typescript
import { VSCodeLMIntegration } from 'poml-vscode/providers/vscodeLMProvider';

// Check if VS Code LM should be used
if (VSCodeLMIntegration.shouldUseVSCodeLM(settings)) {
// Stream responses from VS Code LM
const stream = VSCodeLMIntegration.createStream(messages, settings);

for await (const chunk of stream) {
console.log(chunk);
}
}
```

## API Reference

### Commands

- `poml.detectVSCodeModels` - Detect available VS Code Language Models
- `poml.autoConfigureLM` - Automatically configure language model settings

### Settings

- `poml.languageModel.provider` - Set to "vscode" to use VS Code LM API
- `poml.languageModel.model` - Model identifier (e.g., "copilot/gpt-4o")
- `poml.languageModel.temperature` - Response randomness (0.0-1.0)
- `poml.languageModel.maxTokens` - Maximum response length

## Support

For issues or questions about VS Code LM integration:
1. Check the [POML GitHub Issues](https://github.com/microsoft/poml/issues)
2. Review [VS Code Language Model API docs](https://code.visualstudio.com/api/extension-guides/language-model)
3. Verify your GitHub Copilot subscription is active
25 changes: 25 additions & 0 deletions examples/108_code_review.poml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<!-- POML example for code review and security analysis -->
<poml>
<role>You are an experienced code reviewer specializing in security, performance, and best practices.</role>

<task>Review the provided Python code for security vulnerabilities and improvement opportunities.</task>

<!-- Include the sample code file -->
<document src="assets/108_sample_code.py" />

<instructions>
Focus on:
- SQL injection and XSS vulnerabilities
- Authentication and authorization issues
- Performance bottlenecks
- Code maintainability
</instructions>

<output-format>
Provide a structured code review with:
1. Critical security issues
2. Performance improvements
3. Best practice recommendations
4. Positive observations
</output-format>
</poml>
Loading