Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test - ReportsSimpleTaskParameters #10956

Open
maridematte opened this issue Nov 8, 2024 · 2 comments · Fixed by #10959
Open

Flaky test - ReportsSimpleTaskParameters #10956

maridematte opened this issue Nov 8, 2024 · 2 comments · Fixed by #10959
Labels
flaky-test Tests that have failed but passed on rerun. Priority:1 Work that is critical for the release, but we could probably ship without triaged

Comments

@maridematte
Copy link
Contributor

I've noticed on our pipelines we have a test that fails at random, it is frequent enough to bother during PRs.

The failing test is: Microsoft.Build.BuildCheck.UnitTests.TaskInvocationCheckDataTests.ReportsSimpleTaskParameters during a Windows Full run of our PR pipeline. It also seems to fail during test clean-up and not at the test itself.

Stack trace

at Microsoft.Build.UnitTests.BuildFailureLogInvariant.AssertInvariant(ITestOutputHelper output) in /_/src/UnitTests.Shared/TestEnvironment.cs:line 533
at Microsoft.Build.UnitTests.TestEnvironment.Cleanup() in src/UnitTests.Shared/TestEnvironment.cs:line 99
at Microsoft.Build.UnitTests.TestEnvironment.Dispose() in src/UnitTests.Shared/TestEnvironment.cs:line 72
at Microsoft.Build.BuildCheck.UnitTests.TaskInvocationCheckDataTests.BuildProject(String taskInvocation) in src\BuildCheck.UnitTests\TaskInvocationAnalysisDataTests.cs:line 109
at Microsoft.Build.BuildCheck.UnitTests.TaskInvocationCheckDataTests.ReportsSimpleTaskParameters() in src\BuildCheck.UnitTests\TaskInvocationAnalysisDataTests.cs:line 115
@maridematte maridematte added the flaky-test Tests that have failed but passed on rerun. label Nov 8, 2024
@rainersigwald
Copy link
Member

This failure is a result of this upstream failure (from attachments):

Build Error File MSBuild_pid-6040_50fe91d894c1450d95a1391f680ee8c4.failure.txt: UNHANDLED EXCEPTIONS FROM PROCESS 6040:
=====================
11/7/2024 9:17:51 PM
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckManagerProvider.BuildCheckManager.ProcessProjectEvaluationStarted(ICheckContext checkContext, String projectFullPath) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckManagerProvider.cs:line 544
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckBuildEventHandler.HandleProjectEvaluationStartedEvent(ProjectEvaluationStartedEventArgs eventArgs) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckBuildEventHandler.cs:line 93
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckBuildEventHandler.<.ctor>b__5_2(BuildEventArgs e) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckBuildEventHandler.cs:line 33
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckBuildEventHandler.HandleBuildEvent(BuildEventArgs e) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckBuildEventHandler.cs:line 59
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckConnectorLogger.EventSource_AnyEventRaised(Object sender, BuildEventArgs e) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckConnectorLogger.cs:line 62
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseAnyEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 388
===================
11/7/2024 9:17:51 PM
Microsoft.Build.Exceptions.InternalLoggerException: The build stopped unexpectedly because of an unexpected logger failure. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckManagerProvider.BuildCheckManager.ProcessProjectEvaluationStarted(ICheckContext checkContext, String projectFullPath) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckManagerProvider.cs:line 544
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckBuildEventHandler.HandleProjectEvaluationStartedEvent(ProjectEvaluationStartedEventArgs eventArgs) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckBuildEventHandler.cs:line 93
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckBuildEventHandler.<.ctor>b__5_2(BuildEventArgs e) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckBuildEventHandler.cs:line 33
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckBuildEventHandler.HandleBuildEvent(BuildEventArgs e) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckBuildEventHandler.cs:line 59
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckConnectorLogger.EventSource_AnyEventRaised(Object sender, BuildEventArgs e) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckConnectorLogger.cs:line 62
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseAnyEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 388
   --- End of inner exception stack trace ---
   at Microsoft.Build.Exceptions.InternalLoggerException.Throw(Exception innerException, BuildEventArgs e, String messageResourceName, Boolean initializationException, String[] messageArgs) in /_/src/Build/Errors/InternalLoggerException.cs:line 253
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseAnyEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 419
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseEvent[TArgs](TArgs buildEvent, ArgsHandler`1 handler, ArgsHandler`1 followUpHandler) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 371
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.Consume(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 273
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.Consume(BuildEventArgs buildEvent, Int32 sinkId) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 216
   at Microsoft.Build.BackEnd.Logging.EventRedirectorToSink.Microsoft.Build.Framework.IEventRedirector.ForwardEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventRedirectorToSink.cs:line 57
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckForwardingLogger.EventSource_AnyEventRaised(Object sender, BuildEventArgs buildEvent) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckForwardingLogger.cs:line 56
   at Microsoft.Build.Framework.AnyEventHandler.Invoke(Object sender, BuildEventArgs e)
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseAnyEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 388
===================
11/7/2024 9:17:51 PM
Microsoft.Build.Exceptions.InternalLoggerException: The build stopped unexpectedly because of an unexpected logger failure. ---> Microsoft.Build.Exceptions.InternalLoggerException: The build stopped unexpectedly because of an unexpected logger failure. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckManagerProvider.BuildCheckManager.ProcessProjectEvaluationStarted(ICheckContext checkContext, String projectFullPath) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckManagerProvider.cs:line 544
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckBuildEventHandler.HandleProjectEvaluationStartedEvent(ProjectEvaluationStartedEventArgs eventArgs) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckBuildEventHandler.cs:line 93
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckBuildEventHandler.<.ctor>b__5_2(BuildEventArgs e) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckBuildEventHandler.cs:line 33
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckBuildEventHandler.HandleBuildEvent(BuildEventArgs e) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckBuildEventHandler.cs:line 59
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckConnectorLogger.EventSource_AnyEventRaised(Object sender, BuildEventArgs e) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckConnectorLogger.cs:line 62
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseAnyEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 388
   --- End of inner exception stack trace ---
   at Microsoft.Build.Exceptions.InternalLoggerException.Throw(Exception innerException, BuildEventArgs e, String messageResourceName, Boolean initializationException, String[] messageArgs) in /_/src/Build/Errors/InternalLoggerException.cs:line 253
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseAnyEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 419
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseEvent[TArgs](TArgs buildEvent, ArgsHandler`1 handler, ArgsHandler`1 followUpHandler) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 371
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.Consume(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 273
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.Consume(BuildEventArgs buildEvent, Int32 sinkId) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 216
   at Microsoft.Build.BackEnd.Logging.EventRedirectorToSink.Microsoft.Build.Framework.IEventRedirector.ForwardEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventRedirectorToSink.cs:line 57
   at Microsoft.Build.Experimental.BuildCheck.Infrastructure.BuildCheckForwardingLogger.EventSource_AnyEventRaised(Object sender, BuildEventArgs buildEvent) in /_/src/Build/BuildCheck/Infrastructure/BuildCheckForwardingLogger.cs:line 56
   at Microsoft.Build.Framework.AnyEventHandler.Invoke(Object sender, BuildEventArgs e)
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseAnyEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 388
   --- End of inner exception stack trace ---
   at Microsoft.Build.Exceptions.InternalLoggerException.Throw(Exception innerException, BuildEventArgs e, String messageResourceName, Boolean initializationException, String[] messageArgs) in /_/src/Build/Errors/InternalLoggerException.cs:line 253
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseAnyEvent(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 419
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseEvent[TArgs](TArgs buildEvent, ArgsHandler`1 handler, ArgsHandler`1 followUpHandler) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 371
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.Consume(BuildEventArgs buildEvent) in /_/src/Build/BackEnd/Components/Logging/EventSourceSink.cs:line 273
   at Microsoft.Build.BackEnd.Logging.LoggingService.RouteBuildEvent(BuildEventArgs eventArg) in /_/src/Build/BackEnd/Components/Logging/LoggingService.cs:line 1732
   at Microsoft.Build.BackEnd.Logging.LoggingService.RouteBuildEvent(Object loggingEvent) in /_/src/Build/BackEnd/Components/Logging/LoggingService.cs:line 1687
   at Microsoft.Build.BackEnd.Logging.LoggingService.LoggingEventProcessor(Object loggingEvent) in /_/src/Build/BackEnd/Components/Logging/LoggingService.cs:line 1520
===================

rainersigwald added a commit that referenced this issue Nov 8, 2024
@rainersigwald
Allow concurrent evaluation BuildCheck processing
fb7015a 
Fixes #10956 by moving to a thread-safe dictionary.
@rainersigwald rainersigwald mentioned this issue Nov 8, 2024
Merged
@maridematte
Copy link
Contributor Author

Still happening.

@maridematte maridematte reopened this Nov 19, 2024
@maridematte maridematte added Priority:1 Work that is critical for the release, but we could probably ship without triaged labels Nov 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flaky-test Tests that have failed but passed on rerun. Priority:1 Work that is critical for the release, but we could probably ship without triaged
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow concurrent evaluation BuildCheck processing dotnet/msbuild
2 participants
@rainersigwald @maridematte