Skip to content

TensorDict 0.10.0: MDS, type annotation and typed `MetaData`

Latest
Latest
Compare
Choose a tag to compare
@vmoens vmoens released this 08 Sep 10:08
· 2 commits to main since this release
v0.10.0
c34264a

TensorDict 0.10.0 Release Notes

We are excited to announce the release of TensorDict 0.10.0! This release includes significant improvements to type annotations, new features for metadata handling, enhanced tensor operations, and numerous bug fixes that improve the overall stability and usability of the library.

🎉 Highlights

  • Typed MetaData: Complete rewrite of metadata handling with full type support (#1428)
  • TensorCollection Parent Class: New parent class providing better type annotations and enhanced functionality (#1388)
  • Enhanced String Support: to_struct_array now supports string data types (#1410)
  • Improved Type Safety: Comprehensive type annotation improvements across the entire codebase
  • Better TensorClass Support: Enhanced ClassVar support and super() functionality
  • MDS data interface: the to_mds method creates an MDS dataset on your favourite location -- no more painful columns definition etc (#1426).
  • Support for autograd's grad function (#1417)

✨ New Features

Core Functionality

  • [Feature] Typed MetaData (#1428): Complete rewrite of metadata handling system with full type support, enabling better static analysis and runtime type checking
  • [Feature] TensorCollection parent class and better type annotation (#1388): New parent class that provides enhanced type annotations and improved inheritance hierarchy
  • [Feature] to_struct_array with strings (#1410): Extended to_struct_array functionality to handle string data types
  • [Feature] MDS dataset helper functions (#1426): New helper functions for working with MDS (Multi-Dimensional Scaling) datasets
  • [Feature] implement tensor_split (#1386): Added support for tensor_split operation to match PyTorch tensor API
  • [Feature] accept cap-str as input to set_interaction_type (#1387): Enhanced flexibility in interaction type setting by accepting capitalized strings
  • [Feature] Allow in-place modification of lazy stacks (#1384): Enabled in-place modifications for lazy stacked tensors, improving memory efficiency
  • [Feature] Ensure super() works with TensorClass (#1381): Fixed super() functionality in TensorClass inheritance chains
  • [Feature] Add all everywhere (#1389): Added comprehensive __all__ declarations across all modules for better IDE support and import control

Type System Improvements

  • [Typing] @overload for methods that have a reduce arg (#1427): Added proper type overloads for methods with reduce parameters
  • [BE] A bunch of type annotation improvements (#1409): Comprehensive type annotation improvements across the codebase
  • [BE] Better CompatibleType definition (#1404): Enhanced type definitions for better compatibility checking
  • [BE] Add _from_tensordict to TensorClass (#1403): Added internal method for TensorClass construction from TensorDict
  • [BE] Better type annotation for __getitem__ (#1402): Improved type annotations for indexing operations

🐛 Bug Fixes

Critical Fixes

  • [BugFix] Fix stacking typed MetaData (#1429): Fixed issues with stacking operations on typed metadata
  • [BugFix] Call synchronization when using the td.to("cpu") operation on third-party devices (#1425): Fixed potential precision issues when transferring tensors from third-party devices to CPU
  • [BugFix] Fix missing _maybe_broadcast_other in base.py (#1422): Fixed missing broadcast functionality in base operations
  • [BugFix] lock_() consolidated tds to avoid overriding values (#1408): Fixed value override issues in locked TensorDicts during consolidation

TensorClass Fixes

  • [BugFix] Args for TC with ClassVar (#1401): Fixed argument handling for TensorClass with ClassVar annotations
  • [BugFix] Fix ClassVar support in tensorclass (#1398): Enhanced ClassVar support in tensorclass decorator
  • [BugFix] Fix MetaData assignment in tensorclasses (#1394): Fixed metadata assignment issues in TensorClass instances

Type and API Fixes

  • [BugFix,TypeHint] Fix type annotations in tensorclass stub file (#1421): Fixed type hints in stub files for better IDE support
  • [Bugfix] Fix type annotation for tensordict.keys().iter() (#1413): Fixed iterator type annotations for TensorDict keys
  • [Bugfix] Fix TensorDictModuleWrapper forward (#1415): Fixed forward pass in TensorDictModuleWrapper
  • [Bugfix] Improve various typing issues (#1424): General improvements to typing across the codebase

Tensor Operations

  • [BugFix] repeat_interleave supports tensors (#1391): Fixed repeat_interleave to properly support tensor arguments
  • [BugFix] Fix chunk following split fix (#1377): Fixed chunking operations after split functionality improvements
  • [BugFix] Uneven splits (#1376): Fixed handling of uneven tensor splits
  • [BugFix] JSON/orjson compatibility (#1373): Improved compatibility between JSON and orjson serialization

🔄 Deprecations and Breaking Changes

  • [Deprecation] Upgrade set_list_to_stack behavior (#1382): Updated behavior of set_list_to_stack with proper deprecation warnings for the old API

🛠️ Development and Infrastructure

CI and Testing

  • [CI] Better versioning (#1433): Improved versioning system for better release management
  • [CI] Fix benchmark CI upload with conditional PR testing (#1397): Enhanced CI pipeline for benchmark uploads
  • [CI,Tests] Fix tests (#1396): General test fixes and improvements
  • [CI] Update OSX target (#1378): Updated macOS build targets
  • [CI] Downgrade OSX version in builds (#1375): Adjusted macOS version requirements for broader compatibility

Documentation

  • [Doc,CI] Fix installation of the lib for releases in doc CI (#1432): Fixed library installation in documentation CI
  • [Doc] Fix doc errors (#1431): General documentation error fixes
  • [Doc, CI] Fix Doc CI (#1430): Fixed documentation CI pipeline
  • Fix typos in export tutorial (#1405): Corrected typos in export tutorial documentation

👥 Contributors

Special thanks to all the contributors who made this release possible:

  • Vincent Moens (@vmoens) - Lead maintainer, major features and bug fixes
  • Yichao Zhou (@Yichao-Zhou) - Type system improvements and bug fixes
  • Huazhong (@huazhongyang) - Device synchronization fixes
  • Yoann Poupart (@Xmaster6y) - TensorDictModuleWrapper fixes
  • Chi Zhang (@chz8494) - tensor_split implementation
  • Heon Song (@heonsong) - Documentation improvements

📚 Documentation

For comprehensive documentation, tutorials, and examples, visit:

For a complete list of changes, see the full changelog.

If you encounter any issues, please report them on our GitHub Issues page.

Contributors

chz8494, vmoens, and 4 other contributors
Assets 26
Loading