Skip to content

Commit 245cfd8

Browse files
YunchuWangCopilot
andcommitted
Distinguish already-published NuGet versions from intentional holds
Add alreadyPublishedPackages confirmations that emit only a logged successful no-op, without staging or ESRP submission. Keep skipPackages as intentional holds with precedence over confirmations so existing success, approval, failure and cancellation gates remain fail-closed. Preserve the 19-job graph, root-qualified templates and unchanged publishing branch. Add functional partial-recovery tests and update trial/operator guidance. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c12fa023-523f-4bee-8664-769050035626
1 parent 7eca4a1 commit 245cfd8

4 files changed

Lines changed: 467 additions & 110 deletions

File tree

‎doc/release_process.md‎

Lines changed: 43 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -21,19 +21,25 @@ Its package and assembly name is `Microsoft.DurableTask.LargePayloadPurge.Abstra
2121
standard `Microsoft.DurableTask.*.dll` signing pattern. The existing source traversal, SBOM inclusion,
2222
NuGet signing and per-package approval-gated publication steps apply.
2323

24-
Contract publication also waits for successful Client and Abstractions publication. If either prerequisite
25-
fails, including a duplicate-version upload failure, or is skipped or canceled, contract publication is
26-
skipped rather than treating that result as success.
24+
Contract publication also waits for successful Client and Abstractions jobs. Each prerequisite must
25+
either publish successfully or complete an explicit successful no-op because the operator confirmed its
26+
selected artifact version is already on NuGet.org using `alreadyPublishedPackages`. If either job fails,
27+
including a duplicate-version upload failure, or is intentionally held back, skipped or canceled, contract
28+
publication is skipped rather than treating that result as success.
2729

2830
`Microsoft.DurableTask.Extensions.AzureBlobPayloads` depends on the contract package at every target
2931
framework, so its publication job also waits for successful contract publication (in addition to approval).
30-
If contract publication fails for any reason above, including a duplicate-version upload failure, or is
31-
skipped or canceled, Blob publication is skipped rather than treating that result as success. This extends
32-
the same fail-closed chain: approval, then Client and Abstractions, then the contract, then Blob. A
33-
prerequisite that deliberately skips — for example because its exact version is already published — still
34-
skips the dependent job; this is not a general-purpose publication-idempotency mechanism, just the minimum
35-
ordering these two packages require. Other packages retain their own independent publication jobs gated
36-
only on approval.
32+
If the contract job fails for any reason above, including a duplicate-version upload failure, or is
33+
intentionally held back, skipped or canceled, Blob publication is skipped rather than treating that result
34+
as success. This extends the same fail-closed chain: approval, then Client and Abstractions, then the
35+
contract, then Blob. An already-published confirmation succeeds without staging or submitting that
36+
package, allowing recovery of a partial release: confirm Client/Abstractions to retry the contract and
37+
Blob. For a Blob-only retry, confirm all selected versions already published, including Client,
38+
Abstractions and the contract; their jobs still follow the dependency chain but complete as no-ops.
39+
Intentional holds belong in `skipPackages` and still block dependents. If a package appears in both
40+
lists, the hold takes precedence; it is not permission to publish dependents. Both lists default to
41+
empty and apply only to the selected run. Other packages retain their own independent publication
42+
jobs gated only on approval.
3743

3844
Keep its `RELEASENOTES.md`: `eng/targets/Release.targets` reads it into NuGet package metadata, whereas the
3945
root `CHANGELOG.md` retains repository release history. The shared target also appends a link using the
@@ -131,16 +137,21 @@ After the PR is merged, follow the **Publishing** steps below.
131137
official artifact run is selected.
132138
4. Supply `nuGetOrganization`, the NuGet.org organization confirmed during ESRP NuGet onboarding.
133139
There is no organization default; an unset or unresolved value fails before any ESRP submission.
134-
Set `skipPackages` to an array of exact public package IDs whose selected versions are already
135-
published or are intentionally held back. This is a per-run exclusion, not a permanent package removal.
140+
Set `alreadyPublishedPackages` to exact public package IDs only after verifying their **selected
141+
artifact versions** already exist on NuGet.org. Those jobs log a successful no-op, without package
142+
staging or an ESRP task, so dependent jobs can proceed.
143+
Use `skipPackages` only for intentional holds: those jobs are skipped and still block dependents.
144+
A hold takes precedence if a package is in both lists. These are per-run inputs, not permanent removals.
136145
5. The publisher first stages the signed packages to the existing internal ADO feed. Only after that
137146
succeeds does `nugetApproval` wait for manual approval. The gate displays the selected organization
138-
and requires confirmation of NuGet onboarding, publisher allowlisting and Trusted Publishing.
139-
Do not approve until those prerequisites, the selected artifacts, exclusion list and dependency
147+
and both package lists, and requires confirmation of NuGet onboarding, publisher allowlisting,
148+
Trusted Publishing and each exact already-published version.
149+
Do not approve until those prerequisites, the selected artifacts, both lists and dependency
140150
availability have been checked. Stop at the gate for a validation-only trial.
141151
Rejecting or timing out the gate prevents all public publication.
142-
6. After approval, each selected public package is copied unchanged into its own ESRP folder using
143-
the exact nuspec ID, then submitted with `EsrpRelease@14` and `WaitForReleaseCompletion: true`.
152+
6. After approval, each public package not held or already confirmed published is copied unchanged into
153+
its own ESRP folder using the exact nuspec ID, then submitted with `EsrpRelease@14` and
154+
`WaitForReleaseCompletion: true`.
144155
Internal-only/test packages and symbol packages are not submitted. Missing or multiple matching
145156
packages fail explicitly. Each package retains its independent job and the dependency gates above;
146157
ESRP does not guarantee ordering within a multi-package submission.
@@ -175,9 +186,10 @@ The signed official drop has 18 `.nupkg` files: 14 public stable packages at `1.
175186
`ExportHistory` and `ScheduledTasks` at `1.27.1-preview.2`, the unchanged public
176187
`Microsoft.DurableTask.Generators` at `2.1.0-preview.2`, and internal-only
177188
`Microsoft.DurableTask.InProcessTestHost` at `0.2.3-preview.1`.
178-
For this trial, set `skipPackages: [Microsoft.DurableTask.Generators]` because that exact generator
179-
version is already public. This leaves 16 public targets; the test host has no public job.
180-
The default exclusion list is empty so future generator releases remain enabled.
189+
For this trial, set `alreadyPublishedPackages: [Microsoft.DurableTask.Generators]` and
190+
`skipPackages: []` because that exact generator version is already public. Its job succeeds through a
191+
logged no-op, not an ESRP submission. This leaves 16 public submission targets; the test host has no
192+
public job. Both default lists are empty so future generator releases remain enabled.
181193
Keep `officialPipeline` pinned to the approved signed run `311120` (build number `20261021.2`,
182194
source `2a409da4ac7dcf03c4f25985337eb4623ea6394e`), while the publisher uses the new reviewed and mirrored
183195
source containing this migration (normally the reviewed `main` commit). A run ID and build number
@@ -187,8 +199,9 @@ search indexing alone is not proof that a version has never been published.
187199

188200
#### Pinned trial request and queue permissions
189201

190-
The pipeline's `skipPackages` parameter is a YAML/UI list, for example
191-
`[Microsoft.DurableTask.Generators]`. For the verified REST request through the installed Azure DevOps
202+
The pipeline's `skipPackages` and `alreadyPublishedPackages` parameters are YAML/UI lists. For the
203+
trial, use an empty hold list and `[Microsoft.DurableTask.Generators]` as the already-published list.
204+
For the verified REST request through the installed Azure DevOps
192205
CLI SDK, however, `RunPipelineParameters.template_parameters` is typed as `{str}`: send
193206
`templateParameters` as a map of strings, with the list serialized as JSON **inside a string**.
194207
This request fragment illustrates the wire value; `<confirmed-org>` is a placeholder, not a default:
@@ -197,12 +210,13 @@ This request fragment illustrates the wire value; `<confirmed-org>` is a placeho
197210
{
198211
"templateParameters": {
199212
"nuGetOrganization": "<confirmed-org>",
200-
"skipPackages": "[\"Microsoft.DurableTask.Generators\"]"
213+
"skipPackages": "[]",
214+
"alreadyPublishedPackages": "[\"Microsoft.DurableTask.Generators\"]"
201215
}
202216
}
203217
```
204218

205-
Do not send `skipPackages` as a native JSON array through that client. In the trial, that request
219+
Do not send either list as a native JSON array through that client. In the earlier trial, that request
206220
invalidated or ignored overrides and resolved default source/artifact selections instead.
207221
Verify the resolved publisher source, official artifact run, organization bindings and exclusion
208222
conditions in the actual expanded preview before attempting a run; an HTTP 200 alone is insufficient.
@@ -217,7 +231,7 @@ credentials/APIs, or loosen policies to bypass the denial. This queue-configurat
217231
separate from authorization to use `dtfx-internal-esrp-prod`, ESRP `ESRPRELPACMAN` allowlisting,
218232
NuGet channel onboarding and Trusted Publishing.
219233

220-
#### Observed trial status (2026-10-22)
234+
#### Observed trial status before partial-release recovery fix (2026-10-22)
221235

222236
The tested runtime source is `261b233a828211dae2439afba6a9cf3601d79c5c`. Mirror run `311345`
223237
succeeded and its locked internal ref matched that commit. The actual-source publisher preview
@@ -236,7 +250,11 @@ This is a hard ADO authorization blocker, not a source compilation failure or an
236250
ESRP/backend/NuGet failure. No permission grant, policy change, approval or public distribution was
237251
performed. Post-denial run inventory confirmed that the corrected request created no run and that
238252
`311347` was completed/canceled; no accidental publisher remained active.
239-
Later documentation-only commits do not replace `261b233a` as the exact runtime tested.
253+
Later documentation-only commits did not replace `261b233a` as the exact runtime tested in that preview.
254+
The subsequent source change adds `alreadyPublishedPackages` to distinguish successful no-ops from
255+
intentional holds. It is not covered by that earlier server preview; the final recovery/holdback variants
256+
need their own actual-source compilation before use. The queue-configuration and downstream onboarding
257+
prerequisites remain separate and unresolved; no trial parameter change bypasses them.
240258

241259
For source-only validation, PowerShell 7 with the `Pester` (5.x) and `powershell-yaml` modules can run:
242260

‎eng/publish/nuget-release-steps.yml‎

Lines changed: 76 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -3,84 +3,95 @@ parameters:
33
type: string
44
- name: organizationName
55
type: string
6+
- name: alreadyPublished
7+
type: boolean
8+
default: false
69

710
steps:
8-
- pwsh: |
9-
$ErrorActionPreference = 'Stop'
11+
- ${{ if parameters.alreadyPublished }}:
12+
- pwsh: |
13+
Write-Host "Operator confirmed the selected version of '$env:PACKAGE_ID' is already published on NuGet.org. No package will be staged or submitted to ESRP."
14+
displayName: Already published (${{ parameters.packageId }})
15+
env:
16+
PACKAGE_ID: ${{ parameters.packageId }}
1017
11-
if ([string]::IsNullOrWhiteSpace($env:NUGET_ORGANIZATION) -or $env:NUGET_ORGANIZATION -match '\$\(|\$\{\{') {
12-
throw 'Set nuGetOrganization to the NuGet.org organization confirmed during ESRP NuGet onboarding before publishing.'
13-
}
18+
- ${{ else }}:
19+
- pwsh: |
20+
$ErrorActionPreference = 'Stop'
1421
15-
$packages = @(
16-
foreach ($file in Get-ChildItem -LiteralPath $env:SOURCE_FOLDER -Filter '*.nupkg' -File) {
17-
if ($file.Name -like '*.symbols.nupkg') {
18-
continue
19-
}
22+
if ([string]::IsNullOrWhiteSpace($env:NUGET_ORGANIZATION) -or $env:NUGET_ORGANIZATION -match '\$\(|\$\{\{') {
23+
throw 'Set nuGetOrganization to the NuGet.org organization confirmed during ESRP NuGet onboarding before publishing.'
24+
}
2025
21-
$archive = [System.IO.Compression.ZipFile]::OpenRead($file.FullName)
22-
try {
23-
$nuspecs = @($archive.Entries | Where-Object { $_.FullName -notmatch '[/\\]' -and $_.Name -like '*.nuspec' })
24-
if ($nuspecs.Count -ne 1) {
25-
throw "Expected one root nuspec in '$($file.Name)', found $($nuspecs.Count)."
26+
$packages = @(
27+
foreach ($file in Get-ChildItem -LiteralPath $env:SOURCE_FOLDER -Filter '*.nupkg' -File) {
28+
if ($file.Name -like '*.symbols.nupkg') {
29+
continue
2630
}
2731
28-
$reader = [System.IO.StreamReader]::new($nuspecs[0].Open())
32+
$archive = [System.IO.Compression.ZipFile]::OpenRead($file.FullName)
2933
try {
30-
[xml]$nuspec = $reader.ReadToEnd()
31-
}
32-
finally {
33-
$reader.Dispose()
34-
}
34+
$nuspecs = @($archive.Entries | Where-Object { $_.FullName -notmatch '[/\\]' -and $_.Name -like '*.nuspec' })
35+
if ($nuspecs.Count -ne 1) {
36+
throw "Expected one root nuspec in '$($file.Name)', found $($nuspecs.Count)."
37+
}
3538
36-
if ([string]::IsNullOrWhiteSpace($nuspec.package.metadata.id)) {
37-
throw "Missing package ID in '$($file.Name)'."
38-
}
39+
$reader = [System.IO.StreamReader]::new($nuspecs[0].Open())
40+
try {
41+
[xml]$nuspec = $reader.ReadToEnd()
42+
}
43+
finally {
44+
$reader.Dispose()
45+
}
46+
47+
if ([string]::IsNullOrWhiteSpace($nuspec.package.metadata.id)) {
48+
throw "Missing package ID in '$($file.Name)'."
49+
}
3950
40-
if ($nuspec.package.metadata.id -eq $env:PACKAGE_ID) {
41-
$file
51+
if ($nuspec.package.metadata.id -eq $env:PACKAGE_ID) {
52+
$file
53+
}
54+
}
55+
finally {
56+
$archive.Dispose()
4257
}
4358
}
44-
finally {
45-
$archive.Dispose()
46-
}
47-
}
48-
)
59+
)
4960
50-
if ($packages.Count -ne 1) {
51-
throw "Expected exactly one package for '$env:PACKAGE_ID' in '$env:SOURCE_FOLDER', found $($packages.Count)."
52-
}
61+
if ($packages.Count -ne 1) {
62+
throw "Expected exactly one package for '$env:PACKAGE_ID' in '$env:SOURCE_FOLDER', found $($packages.Count)."
63+
}
5364
54-
if (Test-Path -LiteralPath $env:STAGING_FOLDER) {
55-
throw "ESRP staging folder '$env:STAGING_FOLDER' already exists; refusing to reuse possibly stale content."
56-
}
65+
if (Test-Path -LiteralPath $env:STAGING_FOLDER) {
66+
throw "ESRP staging folder '$env:STAGING_FOLDER' already exists; refusing to reuse possibly stale content."
67+
}
5768
58-
New-Item -ItemType Directory -Path $env:STAGING_FOLDER | Out-Null
59-
Copy-Item -LiteralPath $packages[0].FullName -Destination $env:STAGING_FOLDER
60-
Write-Host "Staged '$($packages[0].Name)' for NuGet organization '$env:NUGET_ORGANIZATION'."
61-
displayName: Stage ${{ parameters.packageId }} for ESRP
62-
env:
63-
PACKAGE_ID: ${{ parameters.packageId }}
64-
NUGET_ORGANIZATION: ${{ parameters.organizationName }}
65-
SOURCE_FOLDER: $(System.DefaultWorkingDirectory)\drop
66-
STAGING_FOLDER: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }}
69+
New-Item -ItemType Directory -Path $env:STAGING_FOLDER | Out-Null
70+
Copy-Item -LiteralPath $packages[0].FullName -Destination $env:STAGING_FOLDER
71+
Write-Host "Staged '$($packages[0].Name)' for NuGet organization '$env:NUGET_ORGANIZATION'."
72+
displayName: Stage ${{ parameters.packageId }} for ESRP
73+
env:
74+
PACKAGE_ID: ${{ parameters.packageId }}
75+
NUGET_ORGANIZATION: ${{ parameters.organizationName }}
76+
SOURCE_FOLDER: $(System.DefaultWorkingDirectory)\drop
77+
STAGING_FOLDER: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }}
6778
68-
- task: SFP.release-tasks.custom-build-release-task.EsrpRelease@14
69-
displayName: ESRP NuGet release (${{ parameters.packageId }})
70-
inputs:
71-
ConnectedServiceName: dtfx-internal-esrp-prod
72-
UseManagedIdentity: true
73-
KeyVaultName: durable-esrp-akv
74-
SignCertName: dts-esrp-cert
75-
ClientId: 0b3ed1a4-0727-4a50-b82a-02c2bd9dec89
76-
DomainTenantId: 33e01921-4d64-4f8c-a055-5bdaffd5e33d
77-
Intent: PackageDistribution
78-
ContentType: NuGet
79-
OrganizationName: ${{ parameters.organizationName }}
80-
ContentSource: Folder
81-
FolderLocation: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }}
82-
WaitForReleaseCompletion: true
83-
Owners: $(Build.RequestedForEmail)
84-
Approvers: $(Build.RequestedForEmail)
85-
ServiceEndpointUrl: https://api.esrp.microsoft.com
86-
MainPublisher: ESRPRELPACMAN
79+
- task: SFP.release-tasks.custom-build-release-task.EsrpRelease@14
80+
displayName: ESRP NuGet release (${{ parameters.packageId }})
81+
inputs:
82+
ConnectedServiceName: dtfx-internal-esrp-prod
83+
UseManagedIdentity: true
84+
KeyVaultName: durable-esrp-akv
85+
SignCertName: dts-esrp-cert
86+
ClientId: 0b3ed1a4-0727-4a50-b82a-02c2bd9dec89
87+
DomainTenantId: 33e01921-4d64-4f8c-a055-5bdaffd5e33d
88+
Intent: PackageDistribution
89+
ContentType: NuGet
90+
OrganizationName: ${{ parameters.organizationName }}
91+
ContentSource: Folder
92+
FolderLocation: $(System.DefaultWorkingDirectory)\esrp\${{ parameters.packageId }}
93+
WaitForReleaseCompletion: true
94+
Owners: $(Build.RequestedForEmail)
95+
Approvers: $(Build.RequestedForEmail)
96+
ServiceEndpointUrl: https://api.esrp.microsoft.com
97+
MainPublisher: ESRPRELPACMAN

0 commit comments

Comments
 (0)