Skip to content

Releases: coursera/dataduct

Version 0.5

26 Jan 00:37
Compare
Choose a tag to compare
  • Cleanup commands being passed in QA steps
  • Add support for postgres
  • Status bar for uploading large files
  • Minor bug fixes

Dataduct v4

19 Nov 17:04
Compare
Choose a tag to compare
  • Support for starting database shell from dataduct cli
  • Fix bug in logger configuration
  • More performance tuning for analyze and vacumm
  • Improved subject line for SNS messages
  • More informed logging for load errors
  • Improvements to decorators
  • PK enforcement changes
  • New load-reload-pk step
  • Support for worker groups
  • Steps to move away from scripts to all code being contained in the library

Dataduct v3

22 Sep 23:38
Compare
Choose a tag to compare
  • More documentation
  • Bug fixes in SQL parser
  • Hooks framework
  • Default bootstrap
  • Teardown
  • Frequency fixes

Dataduct v2

20 Mar 06:27
Compare
Choose a tag to compare

Changes

  • Added Travis integration for continuous builds
  • Added QA steps and logging to S3
  • Added pipeline visualization command
  • Updated Dataduct CLI to a single entry point
  • Added RDS connections for scripts
  • Added bootstrap step for pipelines
  • Added backfill or delay activation
  • Added utput path and input path options
  • Added ability to specify a script directory for transform steps
  • Added SQL sanitization for DBA actions
  • Added SQL parser for select and create table statements
  • Added logging across the library
  • Introduced support for custom steps
  • Added pipeline dependency step
  • Reduced verbosity of imports
  • Step parsing is now isolated in steps
  • Added more examples for steps
  • Added configuration file sync with S3
  • Added config overrides with modes
  • Renamed keywords and safe config failure handling
  • Added EMR streaming support with hadoop 2
  • Cleaned up exceptions
  • Added read the docs support
  • Tables are now created automatically for various steps
  • Added history table support
  • EC2 and EMR config control from YAML
  • Added slack integration
  • Added support for regions in AWS

Version 0.1.0

26 Feb 01:47
Compare
Choose a tag to compare

Just to mark the really old release we had in November.

https://pypi.python.org/pypi/dataduct/0.1.0