Skip to content

Commit 3079e0d

Browse files
authored
Merge branch 'main' into agents/fix-dark-theme-scrollbar-colors
2 parents bfaca54 + 4735247 commit 3079e0d

1,041 files changed

Lines changed: 14740 additions & 4795 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
---
2+
name: update-codex-sdk
3+
description: Update VS Code's bundled @openai/codex dependency to a version available from the private VS Code npm feed, regenerate its protocol client, run the relevant tests, and verify the Codex Agent Host in a launched Code OSS window. Use for bundled Codex SDK/CLI version bumps in the VS Code repository.
4+
---
5+
6+
# Update the bundled Codex SDK
7+
8+
The public npm `latest` version is not necessarily installable in VS Code CI. The Azure pipeline uses the private `vscode` feed, where CFS normally quarantines new third-party package versions for seven days. Select a version from that feed before changing files.
9+
10+
## Select a CI-available version
11+
12+
Run the helper from the repository root:
13+
14+
```bash
15+
node --experimental-strip-types .agents/skills/update-codex-sdk/scripts/latest-private-version.ts
16+
```
17+
18+
It obtains an Azure DevOps access token from the signed-in Azure CLI, queries the same feed used by `build/azure-pipelines/dependencies-check.yml`, and reports the newest stable release for which the root package and every platform binary alias declared by that release are present. It never prints the token. If Azure CLI authentication is missing or expired, ask the user to authenticate rather than falling back to the public registry.
19+
20+
To check a user-requested version explicitly:
21+
22+
```bash
23+
node --experimental-strip-types .agents/skills/update-codex-sdk/scripts/latest-private-version.ts --version 0.149.1
24+
```
25+
26+
Use `--raw` when only the latest complete version string is needed. Do not change the repository or global npm registry merely to probe availability.
27+
28+
## Update every pin
29+
30+
Set `CODEX_VERSION` to the selected exact version. Keep committed lockfile URLs on `https://registry.npmjs.org/`; the private feed determines CI eligibility but is not written into source lockfiles.
31+
32+
```bash
33+
CODEX_VERSION=0.149.1
34+
npm install --save-dev --save-exact --ignore-scripts --registry=https://registry.npmjs.org "@openai/codex@$CODEX_VERSION"
35+
npm --prefix build/agent-sdk/agents/codex install --save-exact --package-lock-only --ignore-scripts --registry=https://registry.npmjs.org "@openai/codex@$CODEX_VERSION"
36+
```
37+
38+
Use `apply_patch` to set `build/codex/codex-version.txt` to the same version, then regenerate the vendored app-server client:
39+
40+
```bash
41+
npm run codex:gen-protocol
42+
```
43+
44+
The expected version-bearing files are:
45+
46+
- `package.json` and `package-lock.json`
47+
- `build/agent-sdk/agents/codex/package.json` and `package-lock.json`
48+
- `build/codex/codex-version.txt`
49+
- `src/vs/platform/agentHost/node/codex/protocol/generated/**`
50+
51+
Never hand-edit generated protocol files. Review their diff, then make the smallest necessary handwritten Agent Host or test changes for protocol additions or type changes. Confirm both package manifests use exact versions and that no private-feed URL entered either lockfile.
52+
53+
## Validate
54+
55+
Run the established checks from the repository root:
56+
57+
```bash
58+
npm run codex:check-protocol
59+
npm run compile
60+
./scripts/test.sh --grep codex
61+
(cd build && npm run test)
62+
npm run test-agent-host-e2e -- --jobs 2
63+
npm run hygiene
64+
```
65+
66+
The Agent Host E2E run exercises the bundled provider SDKs in replay mode. If a Codex SDK change causes replay misses or stale fixtures, read `.github/skills/agent-host-e2e-tests/SKILL.md` before deciding whether to re-record; never weaken or silently skip a failing test.
67+
68+
## Verify a real Codex Agent Host session
69+
70+
After the build and tests pass, read and use `.agents/skills/launch/SKILL.md` to launch an isolated **Agents window** for this checkout. Take a fresh Playwright snapshot, explicitly start a Codex-backed session, and give it a deterministic tool-use task such as:
71+
72+
```text
73+
Run node -p "require('@openai/codex/package.json').version" in this workspace. If it prints <VERSION>, reply exactly CODEX_SDK_<VERSION_WITH_UNDERSCORES>_OK.
74+
```
75+
76+
Success requires all of the following, not merely a window that opened:
77+
78+
- the selected provider is Codex;
79+
- the session invokes the command through the Agent Host and completes;
80+
- the reported version matches every pin;
81+
- the exact sentinel appears in the completed response;
82+
- the Agent Host log shows no startup crash or protocol error.
83+
84+
Save the observed sentinel and test totals for the final report or pull-request description. Follow the launch skill's cleanup steps when finished.
85+
86+
## Deliver
87+
88+
Inspect the complete diff for unrelated changes, secrets, private registry URLs, and generated-file drift. Preserve unrelated user work. Commit, push, or create/update a pull request only when the user has authorized those actions; include the private-feed-selected version and validation evidence in the description.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
interface:
2+
display_name: "Update Codex SDK"
3+
short_description: "Update and verify VS Code's bundled Codex SDK"
4+
default_prompt: "Use $update-codex-sdk to update VS Code to the newest private-feed-available Codex SDK and verify it end to end."
Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
/*---------------------------------------------------------------------------------------------
2+
* Copyright (c) Microsoft Corporation. All rights reserved.
3+
* Licensed under the MIT License. See License.txt in the project root for license information.
4+
*--------------------------------------------------------------------------------------------*/
5+
6+
import { spawnSync } from 'node:child_process';
7+
8+
const feedUrl = 'https://pkgs.dev.azure.com/monacotools/Monaco/_packaging/vscode/npm/registry';
9+
const azureDevOpsResource = '499b84ac-1321-427f-aa17-267ca6975798';
10+
const codexAliasPrefix = 'npm:@openai/codex@';
11+
12+
type OutputFormat = 'text' | 'raw' | 'json';
13+
14+
interface Options {
15+
format: OutputFormat;
16+
requestedVersion: string | undefined;
17+
}
18+
19+
interface CodexVersionMetadata {
20+
optionalDependencies?: Record<string, string>;
21+
}
22+
23+
interface CodexPackument {
24+
versions: Record<string, CodexVersionMetadata>;
25+
}
26+
27+
function usage(): void {
28+
console.error('Usage: node --experimental-strip-types latest-private-version.ts [--raw | --json] [--version <x.y.z>]');
29+
}
30+
31+
function fail(message: string): never {
32+
console.error(`Error: ${message}`);
33+
process.exit(1);
34+
}
35+
36+
function parseArgs(args: string[]): Options {
37+
let format: OutputFormat = 'text';
38+
let requestedVersion: string | undefined;
39+
40+
for (let index = 0; index < args.length; index++) {
41+
const arg = args[index];
42+
if (arg === '--raw') {
43+
format = 'raw';
44+
} else if (arg === '--json') {
45+
format = 'json';
46+
} else if (arg === '--version') {
47+
requestedVersion = args[++index];
48+
if (!requestedVersion) {
49+
usage();
50+
process.exit(2);
51+
}
52+
} else {
53+
usage();
54+
process.exit(2);
55+
}
56+
}
57+
58+
if (requestedVersion && !/^\d+\.\d+\.\d+$/.test(requestedVersion)) {
59+
fail(`--version must be a stable x.y.z version, got ${requestedVersion}`);
60+
}
61+
62+
return { format, requestedVersion };
63+
}
64+
65+
function getAccessToken(): string {
66+
const azureCli = process.platform === 'win32' ? 'az.cmd' : 'az';
67+
const result = spawnSync(azureCli, [
68+
'account',
69+
'get-access-token',
70+
'--resource',
71+
azureDevOpsResource,
72+
'--query',
73+
'accessToken',
74+
'--output',
75+
'tsv',
76+
], { encoding: 'utf8' });
77+
78+
if (result.error) {
79+
fail(`could not run Azure CLI (${result.error.message}). Install it and sign in to the monacotools organization.`);
80+
}
81+
if (result.status !== 0) {
82+
const detail = result.stderr?.trim();
83+
fail(`Azure CLI could not obtain an Azure DevOps token${detail ? `: ${detail}` : ''}`);
84+
}
85+
86+
const token = result.stdout?.trim();
87+
if (!token) {
88+
fail('Azure CLI returned an empty Azure DevOps token. Sign in and try again.');
89+
}
90+
return token;
91+
}
92+
93+
function parseStableVersion(version: string): number[] | undefined {
94+
const match = /^(\d+)\.(\d+)\.(\d+)$/.exec(version);
95+
return match ? match.slice(1).map(Number) : undefined;
96+
}
97+
98+
function compareVersions(left: string, right: string): number {
99+
const leftParts = parseStableVersion(left);
100+
const rightParts = parseStableVersion(right);
101+
if (!leftParts || !rightParts) {
102+
throw new Error('compareVersions only accepts stable versions');
103+
}
104+
for (let index = 0; index < 3; index++) {
105+
const difference = leftParts[index] - rightParts[index];
106+
if (difference !== 0) {
107+
return difference;
108+
}
109+
}
110+
return 0;
111+
}
112+
113+
function binaryVersions(packument: CodexPackument, version: string): string[] {
114+
const optionalDependencies = packument.versions[version]?.optionalDependencies;
115+
if (!optionalDependencies || typeof optionalDependencies !== 'object') {
116+
return [];
117+
}
118+
119+
return Object.values(optionalDependencies)
120+
.filter(value => typeof value === 'string' && value.startsWith(codexAliasPrefix))
121+
.map(value => value.slice(codexAliasPrefix.length));
122+
}
123+
124+
function requiredVersions(packument: CodexPackument, version: string): string[] {
125+
return [version, ...binaryVersions(packument, version)];
126+
}
127+
128+
const { format, requestedVersion } = parseArgs(process.argv.slice(2));
129+
const accessToken = getAccessToken();
130+
const response = await fetch(`${feedUrl}/@openai%2Fcodex`, {
131+
headers: { Authorization: `Bearer ${accessToken}` },
132+
});
133+
134+
if (!response.ok) {
135+
fail(`private VS Code feed returned HTTP ${response.status} ${response.statusText}`);
136+
}
137+
138+
const packument = await response.json() as CodexPackument;
139+
if (!packument || typeof packument !== 'object' || !packument.versions || typeof packument.versions !== 'object') {
140+
fail('private VS Code feed returned an unexpected @openai/codex response');
141+
}
142+
143+
const availableVersions = new Set(Object.keys(packument.versions));
144+
const completeStableVersions = [...availableVersions]
145+
.filter(version => parseStableVersion(version))
146+
.filter(version => binaryVersions(packument, version).length > 0)
147+
.filter(version => requiredVersions(packument, version).every(required => availableVersions.has(required)))
148+
.sort(compareVersions);
149+
150+
const latestVersion = completeStableVersions.at(-1);
151+
if (!latestVersion) {
152+
fail('the private VS Code feed contains no stable Codex release with all platform binaries');
153+
}
154+
155+
const checkedVersion = requestedVersion ?? latestVersion;
156+
if (requestedVersion && availableVersions.has(requestedVersion) && binaryVersions(packument, requestedVersion).length === 0) {
157+
fail(`Codex ${requestedVersion} metadata declares no platform binary aliases, so its availability cannot be verified`);
158+
}
159+
const missingVersions = requiredVersions(packument, checkedVersion).filter(version => !availableVersions.has(version));
160+
const result = {
161+
feed: feedUrl,
162+
latestVersion,
163+
checkedVersion,
164+
available: missingVersions.length === 0,
165+
missingVersions,
166+
};
167+
168+
if (format === 'json') {
169+
console.log(JSON.stringify(result, undefined, 2));
170+
} else if (format === 'raw') {
171+
console.log(latestVersion);
172+
} else if (requestedVersion) {
173+
if (result.available) {
174+
console.log(`Codex ${requestedVersion} is fully available from the private VS Code feed.`);
175+
} else {
176+
console.log(`Codex ${requestedVersion} is not fully available from the private VS Code feed.`);
177+
console.log(`Missing: ${missingVersions.join(', ')}`);
178+
console.log(`Latest fully available stable version: ${latestVersion}`);
179+
}
180+
} else {
181+
console.log(`Latest Codex stable fully available from the private VS Code feed: ${latestVersion}`);
182+
}
183+
184+
if (requestedVersion && !result.available) {
185+
process.exitCode = 1;
186+
}

.github/codeql/codeql-config.yml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Apply this file to CodeQL default setup with the repository property:
2+
# github-codeql-config-file: ./.github/codeql/codeql-config.yml
3+
paths-ignore:
4+
# Keep directory exclusions scoped: product code imports helpers from some
5+
# extension src/test directories.
6+
- 'test/**'
7+
- 'src/**/test/**'
8+
- 'build/**/test/**'
9+
- 'cli/tests/**'
10+
- '.eslint-plugin-local/tests/**'
11+
- 'extensions/*/test/**'
12+
- 'extensions/*/tests/**'
13+
# Keep exact suffixes: themes.test.contribution.ts ships in the product.
14+
- '**/*.test.js'
15+
- '**/*.test.jsx'
16+
- '**/*.test.mjs'
17+
- '**/*.test.cjs'
18+
- '**/*.test.ts'
19+
- '**/*.test.tsx'
20+
- '**/*.test.mts'
21+
- '**/*.test.cts'

.github/workflows/component-fixtures.yml

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@ jobs:
3030
with:
3131
# Need enough history for the merge-base lookup below to succeed even
3232
# when the target branch has advanced since the PR was opened. Full
33-
# clone would be wasteful for this large repo, so cap at 50.
34-
fetch-depth: 50
33+
# clone would be wasteful for this large repo, so cap at 150.
34+
fetch-depth: 150
3535

3636
- name: Setup Node.js
3737
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
@@ -184,8 +184,11 @@ jobs:
184184
if [ "${{ github.event_name }}" = "pull_request" ]; then
185185
# For PRs, diff against the merge-base with the target branch.
186186
TARGET_REF="origin/$BASE_REF"
187-
git fetch --no-tags --depth=50 origin "$BASE_REF"
188-
BASE_SHA=$(git merge-base "$EVENT_SHA" "$TARGET_REF")
187+
git fetch --no-tags --depth=150 origin "$BASE_REF"
188+
if ! BASE_SHA=$(git merge-base "$EVENT_SHA" "$TARGET_REF"); then
189+
echo "::warning::Unable to find a merge base between $EVENT_SHA and $TARGET_REF. The depth-150 shallow history may not contain their common ancestor; skipping screenshot comparison."
190+
exit 0
191+
fi
189192
else
190193
# For push events, diff against the parent commit.
191194
BASE_SHA=$(git rev-parse "$EVENT_SHA^")
@@ -229,6 +232,7 @@ jobs:
229232

230233
- name: Fetch base commit manifest
231234
id: base_manifest
235+
if: steps.base.outputs.base_sha != ''
232236
env:
233237
BASE_SHA: ${{ steps.base.outputs.base_sha }}
234238
run: |
@@ -250,7 +254,7 @@ jobs:
250254
251255
- name: Diff screenshots
252256
id: diff
253-
if: always()
257+
if: always() && steps.base.outputs.base_sha != ''
254258
run: |
255259
node build/lib/screenshotDiffReport.ts \
256260
https://hediet-screenshots.azurewebsites.net \

0 commit comments

Comments
 (0)