v0.0.88

1. Long to Timestamp conversion UDF
2. Deequ version dependency fix
3. Data Quality - Added new data validations to Metorikku based on Amazon Deequ: hasSize, hasUniqueness

v0.0.87

New

1. Data Quality - Added data validations to Metorikku based on Amazon Deequ (IsUnique, IsComplete)
2. LoadIfExist- UDF to attempt to load a table otherwise return an empty DF by specific schema.

v0.0.86

Releasing 0.0.86

v0.0.85

Releasing 0.0.85

v0.0.84

New

1. Added new mode called periodic, basically runs a batch job in a loop
2. Selective merge UDF
3. Added lag reporting to batch and streaming jobs
4. Added empty output protection
5. Support reading metrics from remote location (s3, HDFS)

Improvements

1. Hudi: removeNullColumns flag to remove null columns before writing to hudi
2. Hudi: deletePendingCompactions flag removes pending compactions when running in streaming
3. Hudi: added a manual hive sync mode (this helps hive 1 users)
4. Hudi: added hudiTableName parameter
5. Hive: support hive 2.3
6. Started releasing assembled JAR to maven central

Fixes

1. Hudi: fix metrics reporting when using hudi in streaming
2. Hudi: upgrade to 0.5.2 (breaking change)
3. Refactored build process

v0.0.83

Releasing 0.0.83

v0.0.82

Releasing 0.0.82

v0.0.81

Releasing 0.0.81

v0.0.80

Releasing 0.0.80

v0.0.79

Releasing 0.0.79