Add logging buffering

[evgenyfedorov2]

Related to the #5123 proposal, this PR is focused on the logging buffering only. See also the sampling part #5574.

Microsoft Reviewers: Open in CodeFlow

[dotnet-comment-bot]

‼️ Found issues ‼️

Project	Coverage Type	Expected	Actual
Microsoft.AspNetCore.Diagnostics.Middleware	Line	100	96.03 🔻
Microsoft.AspNetCore.Diagnostics.Middleware	Branch	100	98.51 🔻
Microsoft.Extensions.Telemetry	Line	93	86.2 🔻
Microsoft.Extensions.Telemetry	Branch	93	84.59 🔻
Microsoft.Extensions.Diagnostics.Testing	Line	99	85.78 🔻
Microsoft.Extensions.Diagnostics.Testing	Branch	99	79.57 🔻

🎉 Good job! The coverage increased 🎉
Update MinCodeCoverage in the project files.

Project	Expected	Actual
Microsoft.Extensions.Diagnostics.Probes	70	76
Microsoft.Extensions.Caching.Hybrid	75	86
Microsoft.Extensions.AI.OpenAI	72	77

Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=896358&view=codecoverage-tab

[geeknoid]

Does this hold a global lock while it's serializing? I wonder if this could be a source of latency spikes if there are a few thousand records?

[evgenyfedorov2]

It uses ConcurrentQueue<T>, which is lock-free

[evgenyfedorov2]

Ok, now I have fixed the race condition also mentioned here, but have to add a lock, unfortunately. Unless there is a way to avoid it. If Flush() is called relatively rarely, then the lock almost never blocks, but is still there and wastes some ns per each call

[KalleOlaviNiemitalo]

I don't believe lock (_bufferSwapLock) in GlobalBuffer.Flush() entirely fixes the race condition, if TryEnqueue<TState> is running in parallel. Consider the sequence of events:

1. In TryEnqueue, _activeBuffer.Enqueue(serializedLogRecord) reads _activeBuffer.
2. Flush does

           ConcurrentQueue<SerializedLogRecord> bufferToFlush;
           lock (_bufferSwapLock)
           {
               bufferToFlush = _activeBuffer;
   
               // Swap to the empty standby buffer
               _activeBuffer = _standbyBuffer;
               _activeBufferSize = 0;
   
               // Prepare the new standby buffer for future Flush() calls
               _standbyBuffer = new ConcurrentQueue<SerializedLogRecord>();
           }
   
           SerializedLogRecord[] bufferedRecords = bufferToFlush.ToArray();

   but bufferToFlush does not yet contain the record that TryEnqueue is about to add there.
3. In TryEnqueue, _activeBuffer.Enqueue(serializedLogRecord) finishes.
4. In Flush, bufferToFlush is discarded and the log record is lost.

In addition, there is a race condition on _activeBufferSize, but it is not a problem because it can only add some inaccuracy to the size counting that is already inaccurate for other reasons. (The object size measurement API proposed in dotnet/runtime#24200 might help make it more accurate.)

[evgenyfedorov2]

I don't believe lock (_bufferSwapLock) in GlobalBuffer.Flush() entirely fixes the race condition, if TryEnqueue is running in parallel. Consider the sequence of events:

1. In TryEnqueue, _activeBuffer.Enqueue(serializedLogRecord) reads _activeBuffer.

2. Flush does

           ConcurrentQueue<SerializedLogRecord> bufferToFlush;
           lock (_bufferSwapLock)
           {
               bufferToFlush = _activeBuffer;
   
               // Swap to the empty standby buffer
               _activeBuffer = _standbyBuffer;
               _activeBufferSize = 0;
   
               // Prepare the new standby buffer for future Flush() calls
               _standbyBuffer = new ConcurrentQueue<SerializedLogRecord>();
           }
   
           SerializedLogRecord[] bufferedRecords = bufferToFlush.ToArray();

   but bufferToFlush does not yet contain the record that TryEnqueue is about to add there.

3. In TryEnqueue, _activeBuffer.Enqueue(serializedLogRecord) finishes.

4. In Flush, bufferToFlush is discarded and the log record is lost.

In addition, there is a race condition on _activeBufferSize, but it is not a problem because it can only add some inaccuracy to the size counting that is already inaccurate for other reasons. (The object size measurement API proposed in dotnet/runtime#24200 might help make it more accurate.)

So I have pushed a fix, but it is with a lock. Also added a test to confirm, the test was indeed failing with the quoted code but is not failing with the current version (with locks).