Skip to content

sql-150: reclaim storage metadata for ephemeral items on crash - #38160

Merged
SangJunBak merged 1 commit into
mainfrom
jun/move-temp-to-catalog-split-4
Aug 20, 2026
Merged

sql-150: reclaim storage metadata for ephemeral items on crash#38160
SangJunBak merged 1 commit into
mainfrom
jun/move-temp-to-catalog-split-4

Conversation

@SangJunBak

@SangJunBak SangJunBak commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Motivation

sql-150

Description

catalog: reclaim storage metadata for ephemeral items on crash restart (SQL-150)

When a process crash-restarted, shards from temp tables were never cleaned up. However, on graceful close, we do clean up these shards. This was an existing bug from before the durable temporary objects change.

Verification

Extended the temp object cleanup test to check for shard finalization

@SangJunBak
SangJunBak requested a review from a team as a code owner August 11, 2026 14:25
@SangJunBak SangJunBak added the ci-nightly PR CI control: also trigger Nightly label Aug 11, 2026
@SangJunBak SangJunBak changed the title catalog: reclaim storage metadata for ephemeral items on crash restar… sql-150: reclaim storage metadata for ephemeral items on crash Aug 11, 2026
@linear-code

linear-code Bot commented Aug 11, 2026

Copy link
Copy Markdown

SQL-150

def-
def- previously requested changes Aug 12, 2026
Comment thread test/restart/mzcompose.py
Comment on lines +1483 to +1484
# The comment row dies with its item.
comments = c.sql_query(temp_comment_count, port=6877, user="mz_system")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should actually check something

Suggested change
# The comment row dies with its item.
comments = c.sql_query(temp_comment_count, port=6877, user="mz_system")
# The comment row dies with its item.
comments = c.sql_query(temp_comment_count, port=6877, user="mz_system")
assert comments == [
(0,)
], f"the temp table's comment survived the restart: {comments}"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! 🙈

@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch 3 times, most recently from 5efd494 to 3b9c0fb Compare August 13, 2026 01:35
@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch from 3b9c0fb to 1537f61 Compare August 13, 2026 15:19
@SangJunBak
SangJunBak requested a review from def- August 13, 2026 15:23

@mtabebe mtabebe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, change looks good. Great work

/// Used to reclaim temporary items when the catalog is opened with write
/// intent, at which point every session that could own one is dead.
///
/// This must mirror everything the graceful `Op::DropObjects` path

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch from 1537f61 to 5b6e289 Compare August 14, 2026 16:07
@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch from 5b6e289 to 3636dec Compare August 14, 2026 16:54
@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch 2 times, most recently from 72e7799 to bcfb786 Compare August 17, 2026 19:35
@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch from bcfb786 to 17bf6eb Compare August 19, 2026 19:00
@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch from 17bf6eb to d8ea46b Compare August 19, 2026 20:34
Base automatically changed from jun/move-temp-to-catalog-split-3 to main August 20, 2026 14:46
…t (SQL-150)

When a process crash-restarted, shards from temp tables were never cleaned up. However, on graceful close, we do clean up these shards. This was an existing bug from before the durable temporary objects change.
@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch from d8ea46b to 3dc0e17 Compare August 20, 2026 14:46
@SangJunBak
SangJunBak dismissed def-’s stale review August 20, 2026 18:58

Addressed his changes and he's now on parental leave

@SangJunBak
SangJunBak merged commit c1c744c into main Aug 20, 2026
321 of 323 checks passed
@SangJunBak
SangJunBak deleted the jun/move-temp-to-catalog-split-4 branch August 20, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-nightly PR CI control: also trigger Nightly

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants