Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

codec(ticdc): kafka simple encoding protocol support encoding checkpoint event. #9911

Merged
merged 6 commits into from
Oct 31, 2023

Conversation

3AceShowHand
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #9900

What is changed and how it works?

Check List

Tests

  • Unit test

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

`None`

@ti-chi-bot ti-chi-bot bot added the release-note-none Denotes a PR that doesn't merit a release note. label Oct 17, 2023
@3AceShowHand 3AceShowHand changed the title codec(ticdc): kafka simple encoding protocol support encoding checkpoint event. [DNM]codec(ticdc): kafka simple encoding protocol support encoding checkpoint event. Oct 17, 2023
@ti-chi-bot ti-chi-bot bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 17, 2023
@ti-chi-bot ti-chi-bot bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 18, 2023
@asddongmen
Copy link
Contributor

Please add a issue to track the whole feature and link every related PR to it.

@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 23, 2023
@3AceShowHand 3AceShowHand changed the title [DNM]codec(ticdc): kafka simple encoding protocol support encoding checkpoint event. codec(ticdc): kafka simple encoding protocol support encoding checkpoint event. Oct 23, 2023
@3AceShowHand
Copy link
Contributor Author

/test cdc-integration-storage-test

@3AceShowHand
Copy link
Contributor Author

/test dm-integration-test

@asddongmen asddongmen self-requested a review October 27, 2023 02:30
@3AceShowHand
Copy link
Contributor Author

/retest

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Oct 27, 2023
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Oct 27, 2023

@asddongmen: Your lgtm message is repeated, so it is ignored.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@@ -38,6 +38,7 @@ const (
ProtocolCraft
ProtocolOpen
ProtocolCsv
ProtocolSimple
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use "Simple" as the new protocol name? We will add claim-check large message feature for this protocol, we will add water mark, checkpoint, schema tracking & bootstrap and more other features, can we use a proper name that can represent its functionalities? How about SufficientJson since it is json, but not each message contains the full schema information and is more sufficient than CanalJson?

Copy link
Contributor

@zhangjinpeng87 zhangjinpeng87 Oct 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or ExtensibleJson which means we added many customized feature for this new protocol?

Copy link
Contributor Author

@3AceShowHand 3AceShowHand Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name is not decided yet, use simple temporarily.

Json should not be part of the protocol, since this new protocol does not only support JSON format, we may make it support other encoding formats such as avro, cbor.

Sufficient sounds good to me. Extensible is also a good name, but it's not easy to explain the customized, since this new protocol belongs to the TiCDC itself, all things are customized.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the concrete name, further discussion is required.

@ti-chi-bot ti-chi-bot bot added the lgtm label Oct 31, 2023
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Oct 31, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: asddongmen, nongfushanquan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot removed the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Oct 31, 2023
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Oct 31, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-10-27 09:31:35.873501763 +0000 UTC m=+2599893.460611892: ☑️ agreed by asddongmen.
  • 2023-10-31 07:17:56.332165872 +0000 UTC m=+2937473.919276002: ☑️ agreed by nongfushanquan.

@3AceShowHand
Copy link
Contributor Author

/test dm-integration-test

@ti-chi-bot ti-chi-bot bot merged commit 21e9dfc into pingcap:master Oct 31, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

encoding & decoding checkpoint event
4 participants