vdev_open: clear async remove flag after reopen

[robn]

[Sponsors: Klara, Inc., Wasabi Technology, Inc.]

Motivation and Context

It's possible[*] for a vdev to be flagged for async remove after the pool has suspended. If the removed device has been returned when the pool is resumed, the ASYNC_REMOVE task will still run at the end of txg, and remove the device from the pool again.

[*] Maybe only theoretical. I think to happen the pool would have to suspend, then a different IO would need to return an error, and the media check fail, flagging the async remove. I haven't triggered it on stock OpenZFS. I can trip it regularly on a (not yet published) branch that chains writes and flush IOs, and so is more likely to have outstanding IO after the pool has suspended.

Description

To fix, we clear the async remove flag at reopen, just as we did for the async fault flag in 5de3ac2.

How Has This Been Tested?

ZTS run completed successfully.

I am not really sure how to write a test to prove this (we don't really have the tools), but I think intuitively it makes sense - it matches the existing async fault case.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[amotin]

Seems to make sense. But reminds me #16864 (comment), and makes me think that it would be nice if we would be able to remove (close) vdev even when suspended. In some cases OS may be unable to detach device before it is closed, and in some cases it can not reattach one until it is fully detached.

[allanjude]

Seems to make sense. But reminds me #16864 (comment), and makes me think that it would be nice if we would be able to remove (close) vdev even when suspended. In some cases OS may be unable to detach device before it is closed, and in some cases it can not reattach one until it is fully detached.

It might even make sense to try to close all of the devices on suspend, and reopen them for resume? For this case, but also just to make sure we don't have anything stale, and as you mention, are not typing up any resources.