-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Fix statement store deadlock #9861
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| pub fn maintain(&self) { | ||
| log::trace!(target: LOG_TARGET, "Started store maintenance"); | ||
| let deleted = self.index.write().maintain(self.timestamp()); | ||
| let mut index = self.index.write(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do you need to keep the write lock for the whole duration of this function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
somehow I thought unnamed variable would still be dropped at the end of their "scope" but indeed it is dropped instantly.
Everywhere in the code it seems to lock the index until the operation of the DB is done. But not here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just removed this change then, it is not necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can have this situation:
- index and db have statement s1 in expired.
- maintain starts: index has no expired statements, db still have s1 in expired
- other operation in between: s1 is submitted then s2 is submitted in the same channel as s1: index has s2 as statement, s1 as expired statement, db has s2 as statement and s1 as expired statement
- maintain finishes: remove the statement s1 from expired in db: now index has s2 as statement and s1 as expired, but db only has s2 as statement. They are inconsistent, but it is not harmful AFAICT.
| let mut index = self.index.write(); | ||
| let deleted = index.maintain(self.timestamp()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| let mut index = self.index.write(); | |
| let deleted = index.maintain(self.timestamp()); | |
| let deleted: Vec<_> = { | |
| let deleted = self.index.write().maintain(self.timestamp()); | |
| deleted.into_iter().map(|hash| (col::EXPIRED, hash.to_vec(), None)).collect() | |
| }; | |
| let (purged, active, expired) = { | |
| let index = self.index.read(); | |
| (deleted.len(), index.total_size, index.expired.len()) | |
| }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just removed this change, indeed it is not needed the write lock is dropped instantly so there is no deadlock.
| fn statements(&self) -> Result<Vec<(Hash, Statement)>> { | ||
| let index = self.index.read(); | ||
| let mut result = Vec::with_capacity(index.entries.len()); | ||
| for h in self.index.read().entries.keys() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
|
I would prefer to continue work in #9868 where I can add the other stuff related to statement-store performance |
|
closed in favor of #9868 |
We are doing 2 reads in one time. But this is not good.
Per parking lot doc: