v0.10.0

feat: Max Compute Sink (#52)

* feat: Add Type Info Converter Implementation

* feat: Add Test

* feat: Complete test for MessageTypeInfoConverterTest

* feat: Complete test for TimestampTypeInfoConverter

* feat: Complete test for StructTypeInfoConverter.java

* feat: Complete test for DurationTypeInfoConverter.java

* feat: Complete test for BaseTypeInfoConverterTest.java

* feat: get implementation diff from feat branch

* test: update ConverterOrchestratorTest

* test: Add InsertManagerFactoryTest

* chore: remove unused constructor

* test:
- add DefaultPartitioningStrategyTest
- add TimestampPartitioningStrategyTest
- add PartitioningStrategyFactoryTest

* fix: PartitioningStrategyFactoryTest

* fix: remove unused dependency injection

* test: add non null injected decorator

* test: add test for MaxComputeSchemaCache

* chore: remove unused lombok annotations

* test: Add MaxComputeSinkTest

* test: add test for close

* test: fix typeinfo

* fix: add error handler in MaxComputeSink

* add ProtoUnknownFieldValidationType

* test: Add test for ProtoUnknownFieldValidationType

* fix: use correct converter

* fix: rename method

* chore: rename

* chore: exclude conflicting deps

* chore: fix checkstyle main

* chore: fix checkstyle test

* chore: fix dependencies issue

* chore: remove main

* fix: fix test

* chore: exclude model package and bump version

* feat: add max compute metrics

* feat: add compression option

* fix: fix ordering on instantiation

* feat: Add utils and implement schema update

* feat: Add logging when executing SQL

* chore: checkstyle main

* chore: fix wrong params

* feat: add converter for MaxComputeCompressionAlgorithmConverter

* chore: fix checkstyle

* fix: byte array conversion

* test: fix test

* fix: remove record reordering, put metadata at the beginning

* fix: use timestamp_ntz for proto timestamp and partition

* fix: adjust unit test to use timestamp_ntz instead of timestamp

* chore: remove unused lombok annotations

* chore: exclude maxcompute client and factory from coverage

* chore: exclude maxcompute client from coverage check

* chore: delete unused proto

* test: add test case for log key

* chore: remove unused getters

* test: add compression option test

* chore: remove redundant column field

* fix: duration converter cast to message

* feat: add partition precondition on MaxComputeClient

* feat: add allowSchemaMismatch to false

* chore: remove unused config

* chore: adjust config name

* chore: add sink and config docs

* chore: docs layout

* chore: docs layout + adjust default value of config

* fix: revert mistakenly pushed build gradle change

* chore: fix magic constant

* fix: set null instead of default value to nonexistent field

* chore: fix partitioning strategy

* test: complete unit test

* feat: add configurable date format for timestamp partition key

* chore: change default value and docs

* Maxcompute instrumentation (#55)

* added instrumentation for MC sink

* chore: refactor InsertManager to abstract class

* chore: revert spacing change

* chore: refactor

---------

Co-authored-by: Eka Winata <[email protected]>

* feat: add zone offset for timestamp

* feat: use zone offset config for metadata util

* feat: Use ZoneId instead of ZoneOffset

* feat: use ali auto partitioning strategy [todo fix ProtoDataColumnRecordDecoratorTest]

* test: fix ProtoDataColumnRecordDecoratorTest

* chore: remove unused config

* fix: set MaxComputeCache when partitioningStrategy is not null

* chore: checkstyle

* feat: add checking on sql result

* fix: metadata util

* chore: checkstyle

* fix: add validation for enum based config and make it fail fast on initialization

* chore: fix exception type

* chore: checkstyle

* fix: class naming and add validation for enum value

* chore: Fix MetadataUtil to accept List of TupleString

* chore: switch metadata ordering

* - refactor schema cache to factory method
- fix schema cache to fetch from metadata server

* Fix MaxComputeSchemaCache and its test

* chore: if not exists true table creation parameter

* - checkstyle SinkConfigUtils
- add test for SinkConfigUtils

* - add retry utils
- separate DDL operation to DdlManager

* test: Add test for RetryUtils.java

* feat: add global configs as externalized parameter

* feat: refactor streaming management to StreamingSessionManager class

* feat: externalize partition time unit configuration

* fix: refactor TimestampPartitioningStrategy

* test: add case when passed object is not Record

* chore: remove null checking since message couldn't be null

* chore: add synchronization on schema update method

* chore: checkstyle main

* chore: checkstyle main

* chore: update docs

* chore: update schema docs

* chore: reorder annotation in config

* chore: change version to 0.10.0

* chore: bump aliyun version

* chore: remove redundant enum converter class

* feat: use guava cache for streaming session

* chore: use sessionCache.getUnchecked

* test: update test

* chore: checkstyle

* chore: use const instead of literal string

* chore: wrap update statement with backtick

* test: add IOException use case

* chore: refactor RetryUtils to receive exception predicate

* chore: remove validateConfig()

* fix: make RecordWrapper immutable

* chore: optimize variable declaration on MaxComputeOdpsGlobalSettingsConverter

* fix: remove unused TableTunnel dependencies from InsertManager.java and its implementation

* fix: setup FlushOption on the constructor for InsertManager

* fix: rename static factory method of streaming session manager

* chore: add javadoc for PayloadConverter

* chore: refactor StreamingSessionManager to receive LoadingCache in its constructor

* chore: Refactor PrimitiveTypeInfoConverter to use ImmutableMap

* chore: Rename ConverterOrchestrator.java to ProtobufConverterOrchestrator.java

* chore: refactor name and type info to static var

* chore: move MaxComputeSchemaHelper to root MaxCompute package

* chore: rename getDdlDeclaration to getDDL

* chore: remove builder on MaxComputeSchema

* chore: fix indent

* refactor: MaxComputeSchemaCache to separate handling between non-null and null descriptor

* refactor: use google Sets on PartitioningStrategyFactory

* refactor: make TableValidator limit configurable

* refactor: use immutable map on MetadataUtil

* refactor: remove SinkConfigUtils.java and its test. Transfer the implementation of util method directly in class using it

* chore: add static imports for assertions lib

* chore: rename PayloadConverter.java to ProtobufPayloadConverter

* feat: add validation for timestamp type

* chore: update docs and default value for valid min and max timestamp

* refactor: wrap payload converter to work with DTO

* chore: set default maximum session count to 2 for SINK_MAXCOMPUTE_STREAMING_INSERT_MAXIMUM_SESSION_COUNT

* test: fix assertion

* feat: add timestamp difference validation

* feat: Add NaN and Infinite check float and double

* chore: rename upsertTable to createOrUpdateTable

* chore: refactor inline lambda to static method

* chore: update docs

* fix: use server side schema for building metadata

* chore: add docs for SINK_CONNECTOR_SCHEMA_PROTO_UNKNOWN_FIELDS_VALIDATION

* chore: rename MaxComputeSchemaHelper.java to MaxComputeSchemaBuilder

* refactor: Add method to add uniform errors in SinkResponse and apply it to MaxComputeSink

* feat: parameterized SINK_MAXCOMPUTE_STREAMING_INSERT_TUNNEL_SLOT_COUNT_PER_SESSION

* chore: update generic.md to use more explicit context

* chore: update maxcompute.md to show the correct type mapping

* chore: rename object to parsedObject in ProtoPayload.java

* chore: refactor `convert` method under ProtobufConverterOrchestrator to toMaxComputeTypeInfo and toMaxComputeValue

* chore: add javadoc for ProtobufTypeInfoConverter.java

* chore: checkstyle

* chore: add static import

* chore: remove extraneous space

---------

Co-authored-by: Vaishnavi190900 <[email protected]>

v0.9.2

What's Changed

• chore: upgrade jfrog version by @sumitaich1998 in #47

Full Changelog: v0.9.1...v0.9.2

v0.9.1

What's Changed

• docs: fix docs for kafka metadata columns config by @sumitaich1998 in #44
• fix: allow commas in the template string in json body template by @sumitaich1998 in #45

Full Changelog: v0.9.0...v0.9.1

v0.9.0

What's Changed

• feat: depot httpv2 sink by @sumitaich1998 in #21

Full Changelog: v0.8.2...v0.9.0

v0.8.2

What's Changed

• feat: add metrics for Redis sink by @sumitaich1998 in #39

Full Changelog: v0.8.1...v0.8.2

v0.8.1

Full Changelog: 0.8.0...v0.8.1

What's Changed

• feat: add socket and connection timeout configs in redis sink by @sumitaich1998 in #38

Full Changelog: 0.8.0...v0.8.1

0.8.0

What's Changed

• updating protobuf modules by @ankurs in #34

New Contributors

• @ankurs made their first contribution in #34

Full Changelog: v0.7.4...0.8.0

v0.7.4

What's Changed

• fix: add constant backoff for redis connection issue by @sumitaich1998 in #37

Full Changelog: v0.7.3...v0.7.4

v0.7.3

What's Changed

• fix: temp fix for redis broken connection issue by @sumitaich1998 in #35

Full Changelog: v0.7.2...v0.7.3

v0.7.2

What's Changed

• fix: fetch schema from cache to force refresh when no incoming traffic by @lavkesh in #31
• fix: have a constant delay for schema checker. by @lavkesh in #32

Full Changelog: v0.7.1...v0.7.2