Skip to content

feat(replay): Add checkpoint save and resume functionality for replays - #2152

Closed
bobtista wants to merge 55 commits into
TheSuperHackers:mainfrom
bobtista:bobtista/replay-checkpoint-save
Closed

feat(replay): Add checkpoint save and resume functionality for replays#2152
bobtista wants to merge 55 commits into
TheSuperHackers:mainfrom
bobtista:bobtista/replay-checkpoint-save

Conversation

@bobtista

@bobtista bobtista commented Jan 19, 2026

Copy link
Copy Markdown

Summary

  • Adds ability to save game state checkpoints during replay playback and resume from them
  • Implements command line options for automated checkpoint save/load (-saveAtFrame, -saveTo, -loadCheckpoint)

New Features:

  • Save checkpoints at any frame during replay playback
  • Resume replay from a saved checkpoint with full state restoration
  • Headless mode support for automated testing

CRC Consistency Fixes:

  • Serialize AI, Pathfinder, AIGroup, and PartitionCell state for checkpoint CRC matching
  • Preload CRC from replay file after checkpoint load for verification
  • Restore RNG state at start of update to ensure deterministic behavior
  • Fix shroud state, weapon timing, and path state corruption after checkpoint load

Testing
-[x] Save checkpoint during replay, load it, verify replay continues without desync
-[x]Test headless mode checkpoint save/load via command line
-[x] Verify CRC matches between original replay and checkpoint-resumed replay

TODO

  • Replicate to Generals

Notes

  • Since game state is saved at the checkpoint moment, any changes that would meaningfully affect game state BEFORE that frame will require a new checkpoint being made. Ideally, fixes are about something that happens after the checkpoint and before the mismatch/crash being fixed.

@greptile-apps

greptile-apps Bot commented Jan 19, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds checkpoint save/resume functionality for replay debugging, allowing developers to create save points at specific frames during replay playback and resume from those points in headless mode.

Key changes:

  • Extends RecorderClass to implement Snapshot interface for serialization of replay state (filename, file position, CRC queue)
  • Adds CLI options -saveAtFrame, -saveTo, and -loadCheckpoint for checkpoint management
  • Implements continueReplayFromCheckpoint() to load checkpoint and resume replay from saved position
  • Adds headless mode checks in GameState to prevent UI calls and skip visual-only blocks during checkpoint saves
  • Properly handles CRC state restoration by serializing the CRC queue and associated flags

Implementation notes:

  • When loading a checkpoint, the replay filename is restored from the checkpoint itself (not from -replay CLI arg)
  • Visual blocks (CHUNK_TerrainVisual, CHUNK_TacticalView, CHUNK_ParticleSystem, CHUNK_GhostObject) are skipped in headless saves
  • The save/load system handles missing blocks gracefully, making headless checkpoints compatible with the existing save system
  • CRC queue is properly serialized/restored to maintain sync detection after checkpoint resume

Confidence Score: 5/5

  • This PR is safe to merge with proper testing of checkpoint save/load workflow
  • The implementation is well-designed with proper error handling, memory management (CRCInfo cleanup in reset()), and defensive coding. The serialization logic correctly handles all replay state, and the headless mode guards prevent UI crashes. The save/load system's existing tolerance for missing blocks ensures compatibility.
  • No files require special attention. The core implementation in Recorder.cpp and ReplaySimulation.cpp is solid.

Important Files Changed

Filename Overview
Core/GameEngine/Source/Common/ReplaySimulation.cpp Implemented checkpoint save at specified frame and resume functionality with proper error handling
GeneralsMD/Code/GameEngine/Source/Common/Recorder.cpp Implemented serialization methods for replay state, added file reopening logic, and proper CRCInfo cleanup in reset()
GeneralsMD/Code/GameEngine/Source/Common/System/SaveGame/GameState.cpp Added CHUNK_Recorder snapshot block, headless mode checks for UI calls, and logic to skip visual blocks in headless saves

Sequence Diagram

sequenceDiagram
    participant CLI as Command Line
    participant GM as GameMain
    participant RS as ReplaySimulation
    participant GS as GameState
    participant RC as RecorderClass
    participant GL as GameLogic
    participant FS as FileSystem

    Note over CLI,FS: Checkpoint Save Flow
    CLI->>GM: -replay file.rep -saveAtFrame 49000 -saveTo checkpoint.sav
    GM->>RS: simulateReplays([file.rep])
    RS->>RC: playbackFile(file.rep)
    RC->>FS: Open replay file
    FS-->>RC: File handle
    loop Until target frame or end
        RS->>GL: UPDATE()
        GL-->>RS: Current frame
        alt Frame == saveAtFrame
            RS->>GS: saveGame(checkpoint.sav)
            GS->>RC: xfer(XFER_SAVE)
            Note over RC: Saves replay filename,<br/>file position, CRC state
            GS->>FS: Write checkpoint file
            FS-->>GS: Success
            RS->>RC: stopPlayback()
            RC->>FS: Close replay file
        end
    end

    Note over CLI,FS: Checkpoint Resume Flow
    CLI->>GM: -loadCheckpoint checkpoint.sav
    GM->>RS: continueReplayFromCheckpoint(checkpoint.sav)
    RS->>GS: loadGame(checkpoint.sav)
    GS->>FS: Read checkpoint file
    FS-->>GS: Checkpoint data
    GS->>RC: xfer(XFER_LOAD)
    Note over RC: Restores replay filename,<br/>file position, CRC state
    GS->>RC: loadPostProcess()
    RC->>FS: reopenReplayFileAtPosition(position)
    FS-->>RC: File handle at saved position
    loop Until replay end or CRC mismatch
        RS->>GL: UPDATE()
        GL-->>RS: Current frame
        RC->>FS: Read next commands from position
        alt CRC mismatch detected
            RS-->>GM: Exit with error
        end
    end
    RS-->>GM: Exit code
Loading

@bobtista
bobtista marked this pull request as draft January 19, 2026 21:37
Comment thread GeneralsMD/Code/GameEngine/Source/Common/Recorder.cpp Outdated
Comment on lines +347 to +351
if (TheRecorder->sawCRCMismatch())
{
numErrors++;
break;
}

@Caball009 Caball009 Jan 20, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some issues with the CRC computation for the replay from save game.

@helmutbuhler

Copy link
Copy Markdown

Looks nice! Will -loadCheckpoint also work with savegames that the player generated? If so we could check that savegames from retail can still be loaded.

@bobtista
bobtista force-pushed the bobtista/replay-checkpoint-save branch 7 times, most recently from 646e04a to 67f736d Compare January 20, 2026 23:42
@bobtista
bobtista force-pushed the bobtista/replay-checkpoint-save branch from b6bedd9 to 39210b8 Compare January 31, 2026 00:02
bobtista added 18 commits March 9, 2026 14:15
…clearing pending queue and initializing m_lastCell
bobtista and others added 26 commits March 9, 2026 14:22
…pointer after load

Replace the flag-comparison hack in updateWeaponSet with a proper fix:
- Add syncTemplatePointerAfterLoad() method to WeaponSet
- Call it from Object::loadPostProcess() to sync the pointer
- This ensures m_curWeaponTemplateSet matches what updateWeaponSet would look up

Addresses xezon's review feedback about the original fix being a hack.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The wall pieces loop was using MAX_WALL_PIECES as the array index
instead of i, causing out-of-bounds memory access and potentially
non-deterministic CRC values.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…eckpoint check

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…those with paths

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…s instead of reset

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ckpoint load

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… simulation check

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…l from AI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…vergence after checkpoint load

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…re object updates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… determinism

The m_nextDockActionFrame field was not serialized, causing dock actions
to complete at wrong frames after checkpoint load. This led to premature
state transitions and path computations that diverged from baseline.

Co-Authored-By: bobtista <bobtista@users.noreply.github.com>
…e from AI state machine

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ring defeat events after checkpoint load

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… game

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@bobtista
bobtista force-pushed the bobtista/replay-checkpoint-save branch from b0138d3 to 6473546 Compare March 9, 2026 20:32
@github-actions

github-actions Bot commented Mar 9, 2026

Copy link
Copy Markdown

⚠️ Title/Commit Validation Failed

Invalid commit messages:

  • Fix TurretStateMachine not serializing base StateMachine state
  • Fix DozerActionStateMachine not serializing base StateMachine state
  • cleanup(replay): Remove debug logging added during checkpoint investigation
  • cleanup(replay): Remove obsolete resetTransientStateForCheckpoint workaround
    PR titles and commit messages must follow conventional commits format:
type: Description
type(scope): Description

Allowed types: bugfix, build, chore, ci, docs, fix, feat, perf, refactor, revert, style, test, tweak, unify

See CONTRIBUTING.md for details.

@bobtista

Copy link
Copy Markdown
Author

Closing this for now - the whackamole process was a grind. There's a fix found along the way though that I'll split out into its own PR.

@bobtista bobtista closed this Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants