File stuck in delete queue

[netfenceventu]

Hello.

I've got a problem where a file is seemingly in the delete queue and has been there for months.
This isn't an issue with space (I have plenty of), but "zfs send"ing the incriminated pool produces an overly inflated file (51GB after compression, when it should be around 2GB).

The box is FreeBSD 14.3/amd64, fully patched.

There are two pools, all of then in good HEALTH and with "No known data errors":

# zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
vm      446G   329G   117G        -         -    49%    73%  1.00x    ONLINE  -
zroot  3.56T  1.52T  2.04T        -         -    19%    42%  1.00x    ONLINE  -

#zfs list -o name,mountpoint,canmount,refer,used,usedbychildren,usedbydataset,usedbyrefreservation,usedbysnapshots
zroot                          /zroot                             on          88K  1.52T      1.52T     88K             0B      536K
zroot/ROOT                     none                               on          88K  65.4G      65.4G     88K             0B        0B
zroot/ROOT/default             /                                  noauto    62.1G  65.4G         0B   62.1G             0B     3.23G
zroot/ezjail                   /usr/jails                         on         136K  1.41T      1.41T    136K             0B      328K

As you can see zroot/ROOT/default has 62.1G used by dataset, yet only 3.3G seem to exist when using standard file system tools:

# du -d 0 -x -h /
3.3G    /

With the help of the FreeBSD mailing list, we found out the following:

# zdb -ddd zroot/ROOT/default 0:-1:A |sort -h -b -k 5 |tail -5
    163861    3   128K   128K  77.1M     512   178M  100.00  ZFS plain file
    130966    3   128K   128K  96.4M     512   262M  100.00  ZFS plain file
    163976    3   128K   128K   139M     512   335M  100.00  ZFS plain file
    130956    3   128K   128K   145M     512   343M  100.00  ZFS plain file
       360    3   128K   128K  58.8G     512  90.0G  100.00  ZFS plain file
# zdb -ddddd zroot/ROOT/default 360
Dataset zroot/ROOT/default [ZPL], ID 89, cr_txg 8, 62.1G, 107125 objects, rootbp DVA[0]=<0:103e1032000:1000> DVA[1]=<0:4365c65000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique double size=800L/800P birth=3
7011214L/37011214P fill=107125 cksum=0000000e1765a187:00000fbba696e8af:000aec7540e5bc65:05a9323d03d6cb70

    Object  lvl   iblk   dblk  dsize  dnsize  lsize   %full  type
       360    3   128K   128K  58.8G     512  90.0G  100.00  ZFS plain file
                                               168   bonus  System attributes
        dnode flags: USED_BYTES USERUSED_ACCOUNTED 
        dnode maxblkid: 737498
        path    on delete queue
        uid     0
        gid     0
...

So this is a 58.8GB file that has, from the user perspective, been deleted, but it's still referenced by ZFS.

So the obvious question: how do I get rid of it?
Rebooting (several times) did not help.
Deleting all zroot/ROOT/default's snapshot was useless.

Thanks in advance.

P.S.
In case it is needed:

# zpool get all zroot
NAME   PROPERTY                       VALUE                          SOURCE
zroot  size                           3.56T                          -
zroot  capacity                       42%                            -
zroot  altroot                        -                              default
zroot  health                         ONLINE                         -
zroot  guid                           10368181361075995187           -
zroot  version                        -                              default
zroot  bootfs                         zroot/ROOT/default             local
zroot  delegation                     on                             default
zroot  autoreplace                    off                            default
zroot  cachefile                      -                              default
zroot  failmode                       wait                           default
zroot  listsnapshots                  off                            default
zroot  autoexpand                     off                            default
zroot  dedupratio                     1.00x                          -
zroot  free                           2.04T                          -
zroot  allocated                      1.52T                          -
zroot  readonly                       off                            -
zroot  ashift                         0                              default
zroot  comment                        -                              default
zroot  expandsize                     -                              -
zroot  freeing                        0                              -
zroot  fragmentation                  19%                            -
zroot  leaked                         0                              -
zroot  multihost                      off                            default
zroot  checkpoint                     -                              -
zroot  load_guid                      8455461369367935358            -
zroot  autotrim                       off                            default
zroot  compatibility                  off                            default
zroot  bcloneused                     0                              -
zroot  bclonesaved                    0                              -
zroot  bcloneratio                    1.00x                          -
zroot  feature@async_destroy          enabled                        local
zroot  feature@empty_bpobj            active                         local
zroot  feature@lz4_compress           active                         local
zroot  feature@multi_vdev_crash_dump  enabled                        local
zroot  feature@spacemap_histogram     active                         local
zroot  feature@enabled_txg            active                         local
zroot  feature@hole_birth             active                         local
zroot  feature@extensible_dataset     active                         local
zroot  feature@embedded_data          active                         local
zroot  feature@bookmarks              enabled                        local
zroot  feature@filesystem_limits      enabled                        local
zroot  feature@large_blocks           enabled                        local
zroot  feature@large_dnode            enabled                        local
zroot  feature@sha512                 enabled                        local
zroot  feature@skein                  enabled                        local
zroot  feature@edonr                  disabled                       local
zroot  feature@userobj_accounting     disabled                       local
zroot  feature@encryption             disabled                       local
zroot  feature@project_quota          disabled                       local
zroot  feature@device_removal         enabled                        local
zroot  feature@obsolete_counts        enabled                        local
zroot  feature@zpool_checkpoint       enabled                        local
zroot  feature@spacemap_v2            active                         local
zroot  feature@allocation_classes     disabled                       local
zroot  feature@resilver_defer         disabled                       local
zroot  feature@bookmark_v2            disabled                       local
zroot  feature@redaction_bookmarks    disabled                       local
zroot  feature@redacted_datasets      disabled                       local
zroot  feature@bookmark_written       disabled                       local
zroot  feature@log_spacemap           disabled                       local
zroot  feature@livelist               disabled                       local
zroot  feature@device_rebuild         disabled                       local
zroot  feature@zstd_compress          disabled                       local
zroot  feature@draid                  disabled                       local
zroot  feature@zilsaxattr             disabled                       local
zroot  feature@head_errlog            disabled                       local
zroot  feature@blake3                 disabled                       local
zroot  feature@block_cloning          disabled                       local
zroot  feature@vdev_zaps_v2           disabled                       local
# zfs get all zroot/ROOT/default
NAME                PROPERTY               VALUE                     SOURCE
zroot/ROOT/default  type                   filesystem                -
zroot/ROOT/default  creation               Sun Dec 15 16:58 2019     -
zroot/ROOT/default  used                   65.4G                     -
zroot/ROOT/default  available              1.93T                     -
zroot/ROOT/default  referenced             62.1G                     -
zroot/ROOT/default  compressratio          1.58x                     -
zroot/ROOT/default  mounted                yes                       -
zroot/ROOT/default  quota                  none                      default
zroot/ROOT/default  reservation            none                      default
zroot/ROOT/default  recordsize             128K                      default
zroot/ROOT/default  mountpoint             /                         local
zroot/ROOT/default  sharenfs               off                       default
zroot/ROOT/default  checksum               on                        default
zroot/ROOT/default  compression            lz4                       inherited from zroot
zroot/ROOT/default  atime                  off                       inherited from zroot
zroot/ROOT/default  devices                on                        default
zroot/ROOT/default  exec                   on                        default
zroot/ROOT/default  setuid                 on                        default
zroot/ROOT/default  readonly               off                       local
zroot/ROOT/default  jailed                 off                       default
zroot/ROOT/default  snapdir                hidden                    default
zroot/ROOT/default  aclmode                discard                   default
zroot/ROOT/default  aclinherit             restricted                default
zroot/ROOT/default  createtxg              8                         -
zroot/ROOT/default  canmount               noauto                    local
zroot/ROOT/default  xattr                  on                        default
zroot/ROOT/default  copies                 1                         default
zroot/ROOT/default  version                5                         -
zroot/ROOT/default  utf8only               off                       -
zroot/ROOT/default  normalization          none                      -
zroot/ROOT/default  casesensitivity        sensitive                 -
zroot/ROOT/default  vscan                  off                       default
zroot/ROOT/default  nbmand                 off                       default
zroot/ROOT/default  sharesmb               off                       default
zroot/ROOT/default  refquota               none                      default
zroot/ROOT/default  refreservation         none                      default
zroot/ROOT/default  guid                   13999796839276270198      -
zroot/ROOT/default  primarycache           all                       default
zroot/ROOT/default  secondarycache         all                       default
zroot/ROOT/default  usedbysnapshots        3.23G                     -
zroot/ROOT/default  usedbydataset          62.1G                     -
zroot/ROOT/default  usedbychildren         0B                        -
zroot/ROOT/default  usedbyrefreservation   0B                        -
zroot/ROOT/default  logbias                latency                   default
zroot/ROOT/default  objsetid               89                        -
zroot/ROOT/default  dedup                  off                       default
zroot/ROOT/default  mlslabel               none                      default
zroot/ROOT/default  sync                   standard                  default
zroot/ROOT/default  dnodesize              legacy                    default
zroot/ROOT/default  refcompressratio       1.56x                     -
zroot/ROOT/default  written                1.16M                     -
zroot/ROOT/default  logicalused            103G                      -
zroot/ROOT/default  logicalreferenced      96.7G                     -
zroot/ROOT/default  volmode                default                   default
zroot/ROOT/default  filesystem_limit       none                      default
zroot/ROOT/default  snapshot_limit         none                      default
zroot/ROOT/default  filesystem_count       none                      default
zroot/ROOT/default  snapshot_count         none                      default
zroot/ROOT/default  snapdev                hidden                    default
zroot/ROOT/default  acltype                nfsv4                     default
zroot/ROOT/default  context                none                      default
zroot/ROOT/default  fscontext              none                      default
zroot/ROOT/default  defcontext             none                      default
zroot/ROOT/default  rootcontext            none                      default
zroot/ROOT/default  relatime               on                        default
zroot/ROOT/default  redundant_metadata     all                       default
zroot/ROOT/default  overlay                on                        default
zroot/ROOT/default  encryption             off                       default
zroot/ROOT/default  keylocation            none                      default
zroot/ROOT/default  keyformat              none                      default
zroot/ROOT/default  pbkdf2iters            0                         default
zroot/ROOT/default  special_small_blocks   0                         default
zroot/ROOT/default  snapshots_changed      Sun Oct  5 17:00:01 2025  -
zroot/ROOT/default  prefetch               all                       default
zroot/ROOT/default  autobackup:auto_zroot  true                      inherited from zroot