Skip to content

[BUG] TensorDict.memmap fails for path-like keys #1440

@egorchakov

Description

@egorchakov

Describe the bug

.memmap fails on a TensorDict with path-like keys (e.g. containing / on *nix)

To Reproduce

# file: tensordict_memmap_issue.py

from tempfile import TemporaryDirectory
import torch
from tensordict import TensorDict


def main() -> None:
    td = TensorDict({"a": torch.zeros(10)})
    with TemporaryDirectory() as tmpdir:
        td.memmap(tmpdir)  # succeeds

    td = TensorDict({"a/b/c": torch.zeros(10)})
    with TemporaryDirectory() as tmpdir:
        td.memmap(tmpdir)  # fails


if __name__ == "__main__":
    main()
uv run --isolated --python 3.13 --with torch==2.8.0,tensordict==0.10.0 tensordict_memmap_issue.py
Traceback (most recent call last):
  File "/Users/evgenii/dev/tensordict_memmap_issue.py", line 17, in <module>
    main()
    ~~~~^^
  File "/Users/evgenii/dev/tensordict_memmap_issue.py", line 13, in main
    td.memmap(tmpdir)  # fails
    ~~~~~~~~~^^^^^^^^
  File "/Users/evgenii/.cache/uv/archive-v0/IFrn6zo5IgF1ZlKrCxPZX/lib/python3.13/site-packages/tensordict/base.py", line 6282, in memmap
    return self._memmap_(
           ~~~~~~~~~~~~~^
        prefix=prefix,
        ^^^^^^^^^^^^^^
    ...<6 lines>...
        existsok=existsok,
        ^^^^^^^^^^^^^^^^^^
    ).lock_()
    ^
  File "/Users/evgenii/.cache/uv/archive-v0/IFrn6zo5IgF1ZlKrCxPZX/lib/python3.13/site-packages/tensordict/_td.py", line 2825, in _memmap_
    _populate_memmap(
    ~~~~~~~~~~~~~~~~^
        dest=dest,
        ^^^^^^^^^^
    ...<5 lines>...
        existsok=existsok,
        ^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/evgenii/.cache/uv/archive-v0/IFrn6zo5IgF1ZlKrCxPZX/lib/python3.13/site-packages/tensordict/_td.py", line 4784, in _populate_memmap
    memmap_tensor = MemoryMappedTensor.from_tensor(
        value.data if value.requires_grad else value,
    ...<4 lines>...
        existsok=existsok,
    )
  File "/Users/evgenii/.cache/uv/archive-v0/IFrn6zo5IgF1ZlKrCxPZX/lib/python3.13/site-packages/tensordict/memmap.py", line 250, in from_tensor
    result = torch.from_file(
        str(filename),
    ...<4 lines>...
        device=torch.device("cpu"),
    )
RuntimeError: unable to open file </var/folders/6c/9tvxzb9x2q74bbfkh1jxtby40000gn/T/tmpsjh7q2xq/a/b/c.memmap> in read-write mode: No such file or directory (2)

Expected behavior

Saving/loading works.

System info

import tensordict, numpy, sys, torch
print(tensordict.__version__, numpy.__version__, sys.version, sys.platform, torch.__version__)
0.10.0 2.3.3 3.13.7 (main, Aug 28 2025, 17:02:04) [Clang 20.1.4 ] darwin 2.8.0

Checklist

  • I have checked that there is no similar issue in the repo (required)
  • I have read the documentation (required)
  • I have provided a minimal working example to reproduce the bug (required)

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions