Issue: The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly | related to all OS Versions above 12

[Liwindo]

Describe the issue you are experiencing

The already opened issue #109343 has very strong indication that the root cause is related to the OS as on V12 no issue occurres at all, but with all above, incl. 17, it causes unclean shutdowns in various scenarios & setups. I could de facto not even copy the requested information as it stays online for a few seconds as rebotts itself afterwards again - and that endlessly.
Only solution for me:

1. Reinstall
2. Downgrade to 12
3. Restore of Backup

Due to the n-3 rule you estanblished currently no further updates can be received with OS V12 why I kindly request to priortize this issue as it leaves many of os with an outdated system left.

What operating system image do you use?

rpi4-64 (Raspberry Pi 4/400 64-bit OS)

What version of Home Assistant Operating System is installed?

12

Did the problem occur after upgrading the Operating System?

Yes

Hardware details

Raspberry 4 but also other Hardware setups are affected-

Steps to reproduce the issue

Unknown as several diffent people face the same issue but with most probably different setups incl. Apps.

Anything in the Supervisor logs that might be useful for us?

Couldn't copy due to immediate reboot cycle

Anything in the Host logs that might be useful for us?

Couldn't copy due to immediate reboot cycle

System information

No response

Additional information

No response

[sousekd]

Just adding another data point.

I see the same log entry after every clean Shutdown > Wait > Boot on a brand new Home Assistant Green device updated to:

• Core: 2026.1.2
• Supervisor: 2026.01.1
• Operating System: 16.3
• Frontend: 20260107.2

I did succesfully "factory reset" the Home Assistant Green device before. Only "Zigbee Home Automation", "Internet Printing Protocol (IPP)" and "Google Cast" integrations added to HA. and "Home Assistant Connect ZBT-2" connected.

The log entry does not appear after using the "Restart Home Assistant" option from menu, only after choosing "Shut down system" option, and starting up again.

[Liwindo]

The Watchdog change definitly helps for system stability, less reboots than before. But unfortunatly under high CPU load situations which I intentionally triggered the system still reboots itself with "The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly" error message.

[Liwindo]

Root cause analysis (AI-assisted research, Claude by Anthropic)

After cross-referencing both issues, Claude (Anthropic's AI assistant) identified the problem as a race condition between the Docker stop timeout and the HA Recorder's shutdown sequence, introduced in HAOS 13+ due to the combined effect of:

• Linux kernel upgrade (6.1 → 6.6)
• Docker 27 tightened container stop handling
• Systemd drop-in relocation changing shutdown ordering

What happens

When HA shuts down, the Recorder needs time to checkpoint the SQLite WAL and write the closing timestamp to RecorderRuns. Docker's default stop timeout (10 seconds) now expires before this finishes on slower hardware (e.g. RPi 4). The process gets SIGKILL'd, leaving RecorderRuns.end = NULL and the WAL uncheckpointed — which is exactly what triggers the warning. In severe cases the watchdog detects this and causes a reboot loop.

Why only HAOS 13+ and not v12

The timing window was sufficient on HAOS 12. The changes in 13+ shifted it just enough to break it consistently.

Suggested fixes (two independent layers)

HAOS side: Increase the stop timeout for the homeassistant container in the Supervisor (from 10 s to e.g. 60 s), or extend TimeoutStopSec for docker.service via a systemd drop-in.

Core/Recorder side: Register a SIGTERM handler that flushes and closes the SQLite DB early — before the async shutdown queue — and issue PRAGMA wal_checkpoint(TRUNCATE) as the first step of Recorder teardown.

Workaround for users stuck in a reboot loop

sqlite3 /config/home-assistant_v2.db \
  "UPDATE recorder_runs SET end=datetime('now') WHERE end IS NULL;"

---

Note

This analysis was produced with the help of Claude (Anthropic's AI assistant), which cross-referenced both issues and the relevant source code and release history. It has not been independently verified by a developer — but it may serve as a useful starting point for someone who can.

This issue affects multiple hardware platforms and has been open since 2024.1.6. It is blocking users on HAOS 12 from receiving further updates due to the n-3 support rule. Prioritization would be greatly appreciated.

[agners]

But unfortunatly under high CPU load situations which I intentionally triggered the system still reboots itself

Why would a high CPU load situation trigger a system reboot? 🤔 That seems wrong to begin with.

In general, the system reboot, triggered via Settings > System > Top right power icon > Advanced Options > Reboot system, or reboots triggered by a operating system update, are coordinated by Supervisor, which first shuts down the Core container (and others) before triggering the system shutdown. And the timeout used for that is learned from the Core container itself, the S6_SERVICES_GRACETIME environment variable is by default set to 240000, which is 4 minutes. There should be enough time for a graceful shutdown really.

Unless you trigger the reboot somehow differently. Or in case some custom component is blocking shutdown. It is certainly worth trying safe mode to see if the problem still persists.

[Baxxy13]

I frequently see the warning when I shut down/restart my HA-OS VM via the Proxmox UI.

I also opened a ticket about this, but unfortunately there was no response within six months and it was closed by the bot.
#3760

[agners]

I frequently see the warning when I shut down/restart my HA-OS VM via the Proxmox UI.

Hm, I see that is a problem: The ACPI shutdown initiates a system shutdown, which does not go through the usual Supervisor shutdown procedure. It also isn't patient enough, and simply stops/kills the Home Assistant Core container 🙈

[Baxxy13]

Yeah, that’s how I figured it out too.
A shutdown or reboot of HA-OS that wasn’t initiated via the Supervisor often triggers these warnings.
This is particularly problematic with Proxmox if you’re doing backups using the stop method.

I’m not sure, though, if “my problem” is related to this one, aside from the fact that they share the same warning message.

[Liwindo]

Following your recent changes I did a root cause analysis with Claude and the following is the output. As this goes beyond my level of technical understanding I leave it here and await your feedback if it's bullshit or valid:

Follow-up: Remaining uncovered scenarios after supervisor #6631 + #4577

(Analysis assisted by Claude, Anthropic's AI assistant — not independently
verified by a developer, offered as a starting point for discussion)

@agners — the two PRs together (supervisor #6631 + #4577) clearly address the
primary root cause of external shutdowns bypassing the Supervisor. Thank you
for that — it is the right fix at the right architectural level.

However, after further analysis there appear to be at least two scenarios that
remain uncovered even after those changes land:

---

Scenario 1: Hardware watchdog under sustained high CPU load

If the system becomes genuinely unresponsive, the hardware watchdog can trigger
a hard reset at firmware level — completely outside the reach of logind, systemd,
or the Supervisor. No software-level inhibitor lock can prevent this. This may
explain why high CPU load still causes reboots even with the new handling in place.

---

Scenario 2: Recorder._async_shutdown() / _async_close() hanging indefinitely

This is confirmed by the actual log pattern in related issues
home-assistant/core#109343 and home-assistant/core#117263:

WARNING: Timed out waiting for final writes to complete
WARNING: Shutdown stage 'final write': still running:
         Recorder._async_shutdown() running at core.py:475
WARNING: Timed out waiting for close event to be processed
WARNING: Shutdown stage 'close': still running:
         Recorder._async_close() running at core.py:467

From reading the code and log evidence, the concern is:

• _async_close() appears to call Thread.join() with no timeout via
  async_add_executor_job. If the recorder thread is stuck mid-purge or
  waiting on a locked DB, this future may never resolve — forcing HA's
  watchdog to abandon it and leaving the DB connection open with the WAL
  uncheckpointed.
• Neither _async_shutdown() nor _async_close() appear to have a
  try/finally fallback, meaning cancellation by the watchdog leaves no
  path to force-close the DB cleanly.

Note: I was unable to read the current recorder/core.py source directly
due to a GitHub rate limit error during analysis, so the above is based on
log line references and search results rather than the live file. It is
entirely possible that mitigations already exist that I could not see.

Question for @bdraco or the recorder maintainers: Is the unbounded
Thread.join() in _async_close() a known issue? If so, is a fix already
planned or in progress? If not, would a PR adding a bounded timeout with a
PRAGMA wal_checkpoint(TRUNCATE) + engine.dispose() fallback be welcome?

---

Relationship to agners' fix

These two issues are independent and both matter:

• supervisor#6631 + logind: Set InhibitDelayMaxSec=300 to allow Supervisor graceful shutdown #4577 solve the OS-level coordination — ensuring the
  Supervisor gets enough time to stop Core gracefully on external shutdown
• The potential core.py issue is about application-level robustness —
  ensuring the Recorder closes cleanly within whatever shutdown window it is
  given, even when a purge or large write is in progress on slow hardware

Even with a 300s logind window, a recorder stuck on an unbounded Thread.join()
could still exhaust that window on slow hardware or with a large database.

Happy to open a separate issue in home-assistant/core if that is more
appropriate than tracking it here.