-
Notifications
You must be signed in to change notification settings - Fork 100
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
.memmap
fails on a TensorDict
with path-like keys (e.g. containing /
on *nix)
To Reproduce
# file: tensordict_memmap_issue.py
from tempfile import TemporaryDirectory
import torch
from tensordict import TensorDict
def main() -> None:
td = TensorDict({"a": torch.zeros(10)})
with TemporaryDirectory() as tmpdir:
td.memmap(tmpdir) # succeeds
td = TensorDict({"a/b/c": torch.zeros(10)})
with TemporaryDirectory() as tmpdir:
td.memmap(tmpdir) # fails
if __name__ == "__main__":
main()
uv run --isolated --python 3.13 --with torch==2.8.0,tensordict==0.10.0 tensordict_memmap_issue.py
Traceback (most recent call last):
File "/Users/evgenii/dev/tensordict_memmap_issue.py", line 17, in <module>
main()
~~~~^^
File "/Users/evgenii/dev/tensordict_memmap_issue.py", line 13, in main
td.memmap(tmpdir) # fails
~~~~~~~~~^^^^^^^^
File "/Users/evgenii/.cache/uv/archive-v0/IFrn6zo5IgF1ZlKrCxPZX/lib/python3.13/site-packages/tensordict/base.py", line 6282, in memmap
return self._memmap_(
~~~~~~~~~~~~~^
prefix=prefix,
^^^^^^^^^^^^^^
...<6 lines>...
existsok=existsok,
^^^^^^^^^^^^^^^^^^
).lock_()
^
File "/Users/evgenii/.cache/uv/archive-v0/IFrn6zo5IgF1ZlKrCxPZX/lib/python3.13/site-packages/tensordict/_td.py", line 2825, in _memmap_
_populate_memmap(
~~~~~~~~~~~~~~~~^
dest=dest,
^^^^^^^^^^
...<5 lines>...
existsok=existsok,
^^^^^^^^^^^^^^^^^^
)
^
File "/Users/evgenii/.cache/uv/archive-v0/IFrn6zo5IgF1ZlKrCxPZX/lib/python3.13/site-packages/tensordict/_td.py", line 4784, in _populate_memmap
memmap_tensor = MemoryMappedTensor.from_tensor(
value.data if value.requires_grad else value,
...<4 lines>...
existsok=existsok,
)
File "/Users/evgenii/.cache/uv/archive-v0/IFrn6zo5IgF1ZlKrCxPZX/lib/python3.13/site-packages/tensordict/memmap.py", line 250, in from_tensor
result = torch.from_file(
str(filename),
...<4 lines>...
device=torch.device("cpu"),
)
RuntimeError: unable to open file </var/folders/6c/9tvxzb9x2q74bbfkh1jxtby40000gn/T/tmpsjh7q2xq/a/b/c.memmap> in read-write mode: No such file or directory (2)
Expected behavior
Saving/loading works.
System info
import tensordict, numpy, sys, torch
print(tensordict.__version__, numpy.__version__, sys.version, sys.platform, torch.__version__)
0.10.0 2.3.3 3.13.7 (main, Aug 28 2025, 17:02:04) [Clang 20.1.4 ] darwin 2.8.0
Checklist
- I have checked that there is no similar issue in the repo (required)
- I have read the documentation (required)
- I have provided a minimal working example to reproduce the bug (required)
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working