v0.2.30
What's Changed
- update tags in pkg metadata by @Borda in #384
- 📝 Update Docs: Merge multiple optimized datasets into one by @bhimrazy in #385
- Fix/large num chunks error by @bhimrazy in #381
- fix: non-deterministic CI test failure by @deependujha in #390
- correct the chunk size by adding header size by @tchaton in #395
- pass storage options to s5cmd by @bhimrazy in #397
- CONTRIBUTING.md for LitData by @deependujha in #391
- Feat: add support for custom cache dir in Streaming Dataset by @bhimrazy in #399
- 📝 docs: specify custom cache directory by @bhimrazy in #405
- Fix broken link for CONTRIBUTING.md by @bhimrazy in #404
- Feat/add support for numpy datatypes in tokensloader by @bhimrazy in #401
- Bump version to 0.2.30 by @bhimrazy in #410
Full Changelog: v0.2.29...v0.2.30