The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1219073	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#122148
1218858	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#121924
1218856	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#121924
1218813	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#119575
1218777	dotnet/runtime	Build native test components	Log	dotnet/runtime#121706
1218762	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#122048
2852870	dotnet-emsdk	Build	Log
2852541	dotnet-dotnet	Build (centOSStreamContainer)	Log
1218702	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122072
2852820	dotnet-maui	Run Helix Tests	Log
2852802	dotnet-maui	Run Integration Tests - Build macOS	Log
1217377	dotnet/runtime	Send to Helix	Log	dotnet/runtime#121800
1218585	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#121380
1218607	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122036
1218527	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#121548
1218480	dotnet/runtime	Send to Helix	Log	dotnet/runtime#121853
2852693	dotnet-dotnet	Build (centOSStreamContainer)	Log
1218408	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122128
1218396	dotnet/runtime	Publish	Log
1218439	dotnet/runtime	Send to Helix	Log
2852657	dotnet-maui	Run Helix Tests	Log
1218381	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#121697
1218436	dotnet/runtime	Build product	Log
1217732	dotnet/runtime	Send to Helix	Log	dotnet/runtime#122116
1218379	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#121697
1218362	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#122136
1217502	dotnet/runtime	Send to Helix	Log	dotnet/runtime#121911
1218357	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#122135
1218354	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122134
1218179	dotnet/runtime	Send to Helix	Log	dotnet/runtime#122132
1218330	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#122119
2852573	dotnet-runtime	Send job to Helix (Unix)	Log
2852547	dotnet-docker-tools	🟪 Generate EOL Data for all images in msdotnetcontainersstaging.azurecr.io	Log
1218247	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#122133
1218257	dotnet/runtime	Send to Helix	Log	dotnet/runtime#121614
1218131	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122125
1218136	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#119432
1218118	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#122048
1218121	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122123
1217597	dotnet/runtime	Send to Helix	Log	dotnet/runtime#121412
1218072	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#121862
1218233	dotnet/sdk	🟣 Run AoT Tests	Log	dotnet/sdk#51984
1218271	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#64433
1218232	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#51935
1218227	dotnet/sdk	🟣 Run AoT Tests	Log	dotnet/sdk#51934
1218242	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#64552
1218220	dotnet/sdk	🟣 Run AoT Tests	Log	dotnet/sdk#51925
1218206	dotnet/sdk	🟣 Run AoT Tests	Log	dotnet/sdk#51847
1218111	dotnet/runtime	Send to Helix	Log	dotnet/runtime#121853
1218068	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122127
1218034	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#122128
1218032	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122128
1218013	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#111586
2852429	dotnet-try	Build	Log
1218149	dotnet/runtime	Send to Helix	Log	dotnet/runtime#122130
1217987	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#122094
1218130	dotnet/sdk	🟣 Run AoT Tests	Log	dotnet/sdk#51978
1217875	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#121973
1217999	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122126
1217971	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#114983
1217937	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122112
2852388	dotnet-maui	Run Integration Tests - Build macOS	Log
1217922	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#121643
2852347	dotnet-dotnet	Build (centOSStreamContainer)	Log
1217916	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#121679
1217838	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#121158
2852351	dotnet-runtime	Send job to Helix (Unix)	Log
1217872	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#122121
1216521	dotnet/runtime	Send to Helix	Log	dotnet/runtime#122076
1217869	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122036
1217892	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122120
2852297	dotnet-docker-tools	🟪 Build Images	Log
2852286	dotnet-efcore	Send job to helix	Log
1217704	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122114
1217669	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122111
2852189	dotnet-dotnet	Copy Binary Report (fedoraContainer)	Log
2852259	dotnet-maui	Run Helix Tests	Log
1217747	dotnet/runtime	Send to Helix	Log
1217627	dotnet/runtime	Publish	Log	dotnet/runtime#121614
1217677	dotnet/runtime	Send tests to Helix (Unix)	Log
1217729	dotnet/runtime	Send to Helix	Log
1217624	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#121906
1217548	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122107
2852202	dotnet-efcore	Send job to helix	Log
1217486	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#122105
2852153	dotnet-dotnet	Checkout dotnet-dotnet@dev/mthalman/installer-tests-1xx-version to s	Log
1217505	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#121072
1217439	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#122081
2852162	dotnet-runtime	Run performance job script	Log
1217451	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122072
2852139	dotnet-efcore	Send job to helix	Log
2852124	dotnet-performance	Send job to Helix (Windows)	Log
1217415	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122036
1217380	dotnet/runtime	Run Trimming Tests	Log	dotnet/runtime#121800
1217351	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#122100
1217349	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122100
2852071	dotnet-performance	Send job to Helix (Windows)	Log
2852062	dotnet-maui	Run Helix Tests	Log
1217352	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#119575
1217321	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#122068
Displaying 100 of 2034 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
92	245	2034

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez