[CORE-8637]: storage: fix race between disk_log_impl::new_segment() and disk_log_impl::close()
#24635
+32
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`disk_log_impl::close()` and `disk_log_impl::remove()` both set `_closed` to `true`. There is an assert in `new_segment()` which checks that the log is not closed under `_segments_rolling_lock`. However, neither `close()` or `remove()` previously respected this lock, which could lead to a race condition and a triggered assert, `"cannot add log segment to closed log"`. Await the `_segments_rolling_lock` in `close()` and `remove()`, and additional indicate it as `broken()` for future waiters before setting `_closed = true` to avoid this race.
WillemKauf
changed the title
storage: fix race between disk_log_impl::new_segment() and disk_log_impl::close()storage: fix race between disk_log_impl::new_segment() and disk_log_impl::close()
Retry command for Build#60020please wait until all jobs are finished before running the slash command |
CI test resultstest results on build#60020
test results on build#60032
|
Retry command for Build#60032please wait until all jobs are finished before running the slash command |
|
@WillemKauf I'm wondering who initiates the segment roll while a log is being closed? Is there a missing higher level locking or bad management of resources? Is it possible than fixing the problem this way will just lead to another class of issues like asserts writing/reading from a closed log? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
disk_log_impl::close()anddisk_log_impl::remove()both set_closedtotrue. There is an assert innew_segment()which checks that the log is not closed under_segments_rolling_lock. However, neitherclose()orremove()previously respected this lock, which could lead to a race condition and a triggered assert,"cannot add log segment to closed log".Await the
_segments_rolling_lockinclose()andremove(), and indicate it asbroken()for future waiters before setting_closed = trueto avoid this race.Backports Required
Release Notes
Bug Fixes
vassert().