Skip to content

[upstream] Grok can claim a file was modified without invoking the edit tool #33

Description

@YadOwO

Summary

Grok Build can finish an implementation request with a successful-sounding completion message even though it never invokes a file-editing tool and leaves the files unchanged.

Environment

  • Grok Build CLI 1.0.5
  • macOS
  • Interactive TUI in a VS Code-family terminal
  • Model: grok-4.6
  • Permission mode: --permission-mode bypassPermissions (also reproduced with yolo/always-approve enabled)

Reproduction

  1. Start Grok Build in a Git repository with file editing allowed.
  2. Ask it to make a concrete change to an existing file.
  3. In a session where the edit succeeds, run /clear (or start a new session).
  4. Ask it to make another concrete file change.
  5. Observe the final response and the repository diff.

Expected behavior

Grok should invoke search_replace (or another write tool), show a successful edit/diff, and only then report completion. If it cannot edit, it should report that the change was not made.

Actual behavior

Grok sometimes only lists, reads, and searches files, then reports that the requested changes were completed. The target files remain byte-for-byte unchanged and git diff is empty.

In one captured failing turn:

  • toolCallCount: 12
  • tools used: list_dir, read_file, grep
  • agentFilesTouched: 0
  • totalFilesTouched: 0
  • errorCount: 0
  • no permission denial
  • the search_replace tool was available in the session resources

The previous turn in the same repository and with the same permission settings did successfully call search_replace and produced a diff.

Impact

The final response is not a reliable indicator that an implementation task was applied. Users may believe code was changed when no change occurred, especially after clearing or starting a new session.

Suggested fix

Before allowing an implementation turn to complete, verify that a write/edit tool actually ran successfully and/or that the requested file content changed. If no edit occurred, keep the turn active or report an explicit failure instead of claiming completion.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions