Snapshot publishing is incorrect and inefficient

[cer]

There are two problems with the current approach:

• Publishing SnapshotOffsetEvent to Kafka doesn't handle the scenario where there are not-yet published Customer/Order events in the database (binlog). Consequently, the SnapshotOffsetEvent might be followed by those events. The view service will consume those events and be confused.

• Publishing Customer/OrderSnapshot events via the database is inefficient. Much better to publish directly to Kafka

I propose the following algorithm:

1. Lock tables
2. Get binlog offset from database - can this be done for MySQL and Process
3. Wait for CDC to process up to that offset - CDC API has an API to retrieve binlog position - at this point because the MESSAGE table is locked there are no more events to be published.
4. Publish all messages directly to Kafka
5. Unlock tables

[dartartem]

I will research