Release 0.2.5

What's Changed

• Remove condition on torch installation by @tchaton in #110
• Bump version 0.2.5 by @tchaton in #111

Full Changelog: v0.2.4...v0.2.5

Release 0.2.4

What's Changed

• Update LitGPT references in README.md by @rasbt in #90
• Don't raise a runtimeError if the downloader doesn't exist. by @tchaton in #98
• Added call to setup function of serializer class to set data format by @vgurev in #96
• Fix map() failing to create dataset when input_dir is None by @awaelchli in #100
• Streamingdataset torch compatibility by @yhl48 in #108
• Move to version 0.2.4 by @tchaton in #109

New Contributors

• @rasbt made their first contribution in #90
• @vgurev made their first contribution in #96
• @awaelchli made their first contribution in #100
• @yhl48 made their first contribution in #108

Full Changelog: v0.2.3...v0.2.4

Release 0.2.3

Full Changelog: v0.2.2...v0.2.3

Release 0.2.2

Couple of tiny fixes.

Release 0.2.1

Release 0.2.1. Minor fixes.

Release 0.2.0

⚡ Welcome to Lightning Data

We developed StreamingDataset to optimize training of large datasets stored on the cloud while prioritizing speed, affordability, and scalability.

Specifically crafted for multi-gpu & multi-node (with DDP, FSDP, etc...), distributed training with large models, it enhances accuracy, performance, and user-friendliness. Now, training efficiently is possible regardless of the data's location. Simply stream in the required data when needed.

The StreamingDataset is compatible with any data type, including images, text, video, audio, geo-spatial, and multimodal data and it is a drop-in replacement for your PyTorch IterableDataset class. For example, it is used by Lit-GPT to pretrain LLMs.

This release marks the first of the release from litdata. From now on, we will track all changes within a CHANGELOG.md file.

Thanks to all contributors.