Release 0.2.34

What's Changed

• Fix the serialization of scalar valued tensors by @enrico-stauss in #431
• Add example on how to filter illegal data by @tchaton in #432

Full Changelog: v0.2.33...v0.2.34

Release 0.2.33

What's Changed

• POC: add tiffile serializer by @robmarkcole in #425
• bump by @robmarkcole in #427

Full Changelog: v0.2.32...v0.2.33

Release 0.2.32

What's Changed

• fix: Add mechanism to inform the user a new version is available by @tchaton in #420
• Version 0.2.32 by @tchaton in #421

Full Changelog: v0.2.31...v0.2.32

Release 0.2.31

What's Changed

• Multithreading function for merge_datasets by @yhl48 in #413
• Change S3Client to use user-provided storage_options even in Studio by @grez72 in #415
• Add missing interruptible argument for creating jobs by @tchaton in #418

New Contributors

• @grez72 made their first contribution in #415

Full Changelog: v0.2.30...v0.2.31

v0.2.30

What's Changed

• update tags in pkg metadata by @Borda in #384
• 📝 Update Docs: Merge multiple optimized datasets into one by @bhimrazy in #385
• Fix/large num chunks error by @bhimrazy in #381
• fix: non-deterministic CI test failure by @deependujha in #390
• correct the chunk size by adding header size by @tchaton in #395
• pass storage options to s5cmd by @bhimrazy in #397
• CONTRIBUTING.md for LitData by @deependujha in #391
• Feat: add support for custom cache dir in Streaming Dataset by @bhimrazy in #399
• 📝 docs: specify custom cache directory by @bhimrazy in #405
• Fix broken link for CONTRIBUTING.md by @bhimrazy in #404
• Feat/add support for numpy datatypes in tokensloader by @bhimrazy in #401
• Bump version to 0.2.30 by @bhimrazy in #410

Full Changelog: v0.2.29...v0.2.30

v0.2.29

What's Changed

• Update PL Data to LitData by @bhimrazy in #382
• Fix: Chunks deletion issue by @deependujha in #375
• Bump version 0.2.29 by @deependujha in #383

Full Changelog: v0.2.28...v0.2.29

v0.2.28

What's Changed

• Revert "Feat: Using fsspec to download files" by @tchaton in #380

Full Changelog: v0.2.27...v0.2.28

v0.2.27

What's Changed

• ci: drop dependabot by @Borda in #361
• azure storage options by @mohanreddypmr in #365
• switch lightning-cloud to lightning SDK by @Borda in #369
• remove not violated bandit rules from ignore by @Borda in #372
• fixing typos in errors & docs by @Borda in #371
• reduce unnecessary pass by @Borda in #373
• fixing docstrings by @Borda in #374
• improve hint readability by @Borda in #376
• Bump version to 0.2.27.dev by @rasbt in #378
• fix import & asignement issue by @Borda in #377
• Feat: Using fsspec to download files by @deependujha in #348
• Bump version to 0.2.27 by @bhimrazy in #379

Full Changelog: v0.2.26...v0.2.27

v0.2.26.dev

What's Changed

• ci: drop dependabot by @Borda in #361
• azure storage options by @mohanreddypmr in #365
• switch lightning-cloud to lightning SDK by @Borda in #369
• remove not violated bandit rules from ignore by @Borda in #372
• fixing typos in errors & docs by @Borda in #371
• reduce unnecessary pass by @Borda in #373
• fixing docstrings by @Borda in #374
• improve hint readability by @Borda in #376
• Bump version to 0.26.dev by @rasbt in #378

Full Changelog: v0.2.26...v0.2.26.dev

v0.2.26

What's Changed

• Bump mosaicml-streaming from 0.8.0 to 0.8.1 by @dependabot in #346
• Adds check for existence of dataset path before loading index file by @bhimrazy in #350
• Bump pypa/gh-action-pypi-publish from 1.9.0 to 1.10.0 by @dependabot in #352
• Bump coverage from 7.5.3 to 7.6.1 by @dependabot in #345
• bump/ci: update to 0.11.7 by @Borda in #355
• Update README.md by @tchaton in #356
• tchaton patch 1 by @tchaton in #357
• Update README.md by @tchaton in #358
• Update README.md by @tchaton in #359
• Fix: Prevent multiple processes from copying the same file when using… by @dallmann-uniwue in #353
• LitData release 0.2.26 by @tchaton in #360

New Contributors

• @dallmann-uniwue made their first contribution in #353

Full Changelog: v0.2.25...v0.2.26