diff --git a/001-cdc/001---cdc.md b/001-cdc/001---cdc.md index 151ef9d..5ac856b 100644 --- a/001-cdc/001---cdc.md +++ b/001-cdc/001---cdc.md @@ -80,15 +80,15 @@ For the subscriber/consumer part, we can try using the [Confluent PostgreSQL sin ### Approach 3: Implement our publisher and subscriber in Go, using the PostgreSQL's logical replication -In this approach, we will implement something similar to Debezium server/engine, but using Golang libraries such as [wal2json](https://github.com/eulerto/wal2json) and [wal-g](https://github.com/wal-g/wal-g). +In this approach, we will implement something similar to Debezium server/engine, but using tools and libraries such as [wal2json](https://github.com/eulerto/wal2json) and [wal-g](https://github.com/wal-g/wal-g). The steps in this approach would be: -1. Using Wal-g, we create a base backup of the source database and restore it to the target database. From this point, the target database must be in read-only mode for users other than our plugin user. +1. Using wal-g, we create a base backup of the source database and restore it to the target database. From this point, the target database must be in read-only mode for users other than our plugin user. 2. Using wal2json, we create a stream of transactions from the source database from the point that the base backup was created. 3. A consumer reads the stream and applies the transactions to the target database. -4. From some point, we determine that the target database is somehow in sync with the source database, by somehow, I mean that there are less than a threshold number of transactions in the queue. At this point, we can say that databases are 'ready' for migration. -5. When databases are 'ready', and the user confirms, we need to put the source database in read-only mode and the consumer should process the remaining transactions until nothing is remaining. +4. At one point, we try to determine whether the target database is in sync with the source database or not, i.e. there are less than a threshold number of transactions in the queue. At this point, we can say that databases are 'ready' for migration. +5. When databases are 'ready', and the user confirms, we need to put the source database in read-only mode and the consumers should process the remaining transactions until all of them are processed. - During the time that the consumer is applying the remaining transaction, and the source DB is in read-only mode, there might be some write queries coming to the database. I can think of two ways to handle these queries: + While the consumers are applying the remaining transactions, and the source DB is in read-only mode, and there might be some write queries coming to the database. I can think of two ways to handle these queries: 1. Reject the queries and return an error to the client. This approach will cause a very short glitch and downtime, but according to the fact that we already know that very few transactions remain, this downtime will be very short.