Support BYOK endpoints for Inline suggestions (fix #318545)

[unbadfish]

TL;DR: Fix #318545 by bridge FIM (Fill in Middle) requests to custom endpoint, thus enjoy "vanilla" copilot edit suggestion in custom endpoint.
See "usage" section to edit your settings. NOTE: read endpoint's doc and use FULL URL in completionsUrl

Problem

Chat already supports BYOK (Bring Your Own Key) models via chatLanguageModels.json, but inline code completions always require GitHub's cloud. Setting github.copilot.selectedCompletionModel to a custom model ID is silently ignored because the ID is not in the list returned by GitHub's API — the extension falls back to the default Copilot cloud model and every completion goes through proxy.business.githubcopilot.com. This makes self-hosted local models (llama.cpp, FastFlowLM, vLLM) and BYOK endpoints cannot use "vanilla" copilot edit suggestion in this situation.

Solution

Reuse the existing chatLanguageModels.json configuration as the single source of truth for BYOK completion models. A model becomes eligible for inline completions when it declares a completionsUrl (a full OpenAI-compatible FIM endpoint, used verbatim). When such a model is selected, the completions pipeline POSTs a FIM request to that endpoint with the user's own API key. Enterprise policy (isClientBYOKAllowed) is inherited automatically, exactly as it is for chat.

Highlight

By bridge FIM requests to custom endpoint, users can enjoy "vanilla" copilot edit suggestion (pop-up window, key bounding, not extra extension, etc.) in custom endpoint.

Changes

• Configuration pipeline — a shared registry parses customendpoint/customoai groups reported by the BYOK chat providers (which receive the decoded configuration, including API keys resolved from secret storage). A small contribution triggers core model resolution via lm.selectChatModels({vendor}) and keeps the completions model manager in sync. One line in the core extHostLanguageModels bridge forwards the group name so multiple groups per vendor coexist.
• Request routing — LiveOpenAIFetcher branches on the selected custom model: the request goes to completionsUrl verbatim with FIM fields only (prompt, suffix, max_tokens, temperature, top_p, n, stop, stream, model); Copilot-only fields (extra, nwo, code_annotations) and headers are stripped. The transport layer omits Copilot-specific headers, sends Authorization: Bearer <apiKey> only when a key is configured.
• Multi-candidate behavior — n is forced to 1 because most OpenAI-compatible FIM endpoints reject n > 1. Cycling still produces multiple candidates: each Alt+] with ≤1 cached candidate issues a fresh request whose sample is merged (deduplicated) with the cache, and the cycling sampling temperature (0.2) is preserved so greedy servers yield different completions per request.
• Selection & schema — github.copilot.selectedCompletionModel accepts the bare id, group/id and vendor/group/id forms; the model picker lists custom models with their group name; package.json schemas for customendpoint/customoai gain completionsUrl (model-level and group-level). The inline completion provider is registered without a Copilot token when BYOK models are configured, enabling signed-out/offline usage.
• Tests — new vitest suite for the parser (9 cases) and new mocha cases for BYOK request construction, header/key handling and error branches. All tests use mocked fetches and vendor-neutral URLs; no network access and no real API keys are involved.

Validation

• tsc --noEmit (extension) and TS7 native --noEmit on tsconfig.json both pass
• vitest: 9/9 passed
• Manual verification against DeepSeek FIM (https://api.deepseek.com/beta/completions): all completion requests route to the configured endpoint with 200 responses; multi-line/single-line stop sequences and Alt+] cycling verified; offline usage verified (no Copilot token required)

Usage

1. add a model with completionsUrl in chatLanguageModels.json.
   NOTE: read endpoint's doc and use FULL URL in completionsUrl.

// chatLanguageModels.json
[
	{
		"name": "deepseek",
		"vendor": "customendpoint",
		"apiKey": "${input:chat.lm.secret.xxxxxxxx}",
		"apiType": "messages",
		"models": [
			{
				"id": "deepseek-v4-flash",
				"name": "DS-v4-flash",
				"url": "https://api.deepseek.com/anthropic",
				"completionsUrl": "https://api.deepseek.com/beta/completions",
				"streaming": true,
				"toolCalling": true,
				"vision": false,
				"thinking": true,
				"contextWindow": 1000000,
				"maxOutputTokens": 384000,
				"supportsReasoningEffort": [
					"low",
					"high",
					"max"
				]
			}
		]
	}
]

2. run command GitHub Copilot: Change Completions Model, select your BYOK model.
3. open a file to trigger FIM completions suggestion.

Notes

• The completionsUrl is used exactly "as is"; URL correctness (e.g. appending something as /beta/completions) is the user's responsibility. READ the provider's DOC!
• Models without a completionsUrl remain chat-only.
• Endpoints that reject n > 1 are fully supported; cycling falls back to repeated single-sample requests.

---

co-author by deepseek-v4-flash&pro, harness by copilot.

[unbadfish]

@microsoft-github-policy-service agree

[Copilot]

Pull request overview

Adds BYOK FIM endpoints to GitHub Copilot inline completions, including model discovery, routing, selection, and offline support.

Changes:

• Bridges configured BYOK models into the completion model registry and picker.
• Routes FIM requests directly to custom endpoints with BYOK authentication.
• Adds schema entries and tests for custom completion models.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/vs/workbench/api/common/extHostLanguageModels.ts	Forwards language-model group metadata.
extensions/copilot/src/platform/nesFetch/node/completionsFetchServiceImpl.ts	Adapts transport headers and quota handling.
extensions/copilot/src/platform/nesFetch/common/completionsFetchService.ts	Extends the fetch contract for custom endpoints.
extensions/copilot/src/extension/completions/vscode-node/completionsCoreContribution.ts	Enables offline BYOK provider registration.
extensions/copilot/src/extension/completions-core/vscode-node/lib/src/openai/test/model.test.ts	Tests BYOK model selection and listing.
extensions/copilot/src/extension/completions-core/vscode-node/lib/src/openai/test/fetch.test.ts	Tests custom FIM request behavior.
extensions/copilot/src/extension/completions-core/vscode-node/lib/src/openai/model.ts	Integrates BYOK models into model resolution.
extensions/copilot/src/extension/completions-core/vscode-node/lib/src/openai/fetch.ts	Constructs and routes custom FIM requests.
extensions/copilot/src/extension/completions-core/vscode-node/lib/src/openai/config.ts	Propagates custom model metadata.
extensions/copilot/src/extension/completions-core/vscode-node/lib/src/ghostText/requestContext.ts	Carries BYOK models through request context.
extensions/copilot/src/extension/completions-core/vscode-node/lib/src/ghostText/ghostText.ts	Adds custom models to ghost-text requests.
extensions/copilot/src/extension/completions-core/vscode-node/lib/src/ghostText/completionsFromNetwork.ts	Adapts BYOK candidate sampling.
extensions/copilot/src/extension/completions-core/vscode-node/extension/src/modelPicker.ts	Lists custom completion models.
extensions/copilot/src/extension/completions-core/vscode-node/extension/src/byokCompletionModelsContribution.ts	Synchronizes configured BYOK models.
extensions/copilot/src/extension/completions-core/vscode-node/completionsServiceBridges.ts	Exposes BYOK bridge setup.
extensions/copilot/src/extension/byok/vscode-node/byokContribution.ts	Clears models when policy disables BYOK.
extensions/copilot/src/extension/byok/vscode-node/abstractLanguageModelChatProvider.ts	Publishes resolved BYOK configuration.
extensions/copilot/src/extension/byok/common/test/byokCompletionModels.spec.ts	Tests BYOK configuration parsing.
extensions/copilot/src/extension/byok/common/byokCompletionModels.ts	Implements the shared BYOK model registry.
extensions/copilot/package.json	Adds completion endpoint schema fields.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[unbadfish]

Sorry for the incorrect conflict handling above. Now I have corrected it.

Commit d6712eb is a merge commit that brings in the cherry-picked PR microsoft/vscode#332316 (commit 2ccb7d1). The merge works correctly — both the supportsUnifiedCompletions model-strategy logic from #332316 and the custom BYOK offline completion models logic are preserved in jointInlineCompletionProvider.ts.

Screenshot:

[Ciel (lfsty)]

Will this inline suggestion support the <|fim_prefix|><|fim_suffix|><|fim_middle|> format? For example, models from the Qwen2.5-Coder series.

[unbadfish]

Will this inline suggestion support the <|fim_prefix|><|fim_suffix|><|fim_middle|> format? For example, models from the Qwen2.5-Coder series.

No. As copilot uses the prompt and suffix parameter to mark the prefix and suffix (not in prompt), this PR only supports the following situation: the model supports prefix & suffix model AND the provider supports the prompt and suffix parameter in request (root, not in extra_body).
e.g. For siliconflow, this PR only supports https://api.siliconflow<.cn or .com>/v1/completions, not https://api.siliconflow<.cn or .com/v1/chat/completions. See "Using the completions Interface" in https://docs.siliconflow.com/en/userguide/guides/fim#2-2-using-the-completions-interface .

[Ciel (lfsty)]

Will this inline suggestion support the <|fim_prefix|><|fim_suffix|><|fim_middle|> format? For example, models from the Qwen2.5-Coder series.

No. As copilot uses the prompt and suffix parameter to mark the prefix and suffix (not in prompt), this PR only supports the following situation: the model supports prefix & suffix model AND the provider supports the prompt and suffix parameter in request (root, not in extra_body). e.g. For siliconflow, this PR only supports https://api.siliconflow<.cn or .com>/v1/completions, not https://api.siliconflow<.cn or .com/v1/chat/completions. See "Using the completions Interface" in https://docs.siliconflow.com/en/userguide/guides/fim#2-2-using-the-completions-interface .

Thanks for the clear and detailed response.