Releases: coursera/dataduct
Releases · coursera/dataduct
Version 0.5
Dataduct v4
- Support for starting database shell from dataduct cli
- Fix bug in logger configuration
- More performance tuning for analyze and vacumm
- Improved subject line for SNS messages
- More informed logging for load errors
- Improvements to decorators
- PK enforcement changes
- New load-reload-pk step
- Support for worker groups
- Steps to move away from scripts to all code being contained in the library
Dataduct v3
- More documentation
- Bug fixes in SQL parser
- Hooks framework
- Default bootstrap
- Teardown
- Frequency fixes
Dataduct v2
Changes
- Added Travis integration for continuous builds
- Added QA steps and logging to S3
- Added pipeline visualization command
- Updated Dataduct CLI to a single entry point
- Added RDS connections for scripts
- Added bootstrap step for pipelines
- Added backfill or delay activation
- Added utput path and input path options
- Added ability to specify a script directory for transform steps
- Added SQL sanitization for DBA actions
- Added SQL parser for select and create table statements
- Added logging across the library
- Introduced support for custom steps
- Added pipeline dependency step
- Reduced verbosity of imports
- Step parsing is now isolated in steps
- Added more examples for steps
- Added configuration file sync with S3
- Added config overrides with modes
- Renamed keywords and safe config failure handling
- Added EMR streaming support with hadoop 2
- Cleaned up exceptions
- Added read the docs support
- Tables are now created automatically for various steps
- Added history table support
- EC2 and EMR config control from YAML
- Added slack integration
- Added support for regions in AWS
Version 0.1.0
Just to mark the really old release we had in November.