Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kafka(ticdc): support columns dispatcher. #9863

Merged
merged 25 commits into from
Oct 17, 2023
Merged

kafka(ticdc): support columns dispatcher. #9863

merged 25 commits into from
Oct 17, 2023

Conversation

3AceShowHand
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #9862

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note 

kafka sink support columns dispatcher to allow user to specify columns to calculate the partition number

@ti-chi-bot ti-chi-bot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 11, 2023
@3AceShowHand
Copy link
Contributor Author

/test cdc-integration-storage-test

@3AceShowHand
Copy link
Contributor Author

/test dm-integration-test

@3AceShowHand
Copy link
Contributor Author

/test cdc-integration-kafka-test

@3AceShowHand
Copy link
Contributor Author

/retest

@3AceShowHand
Copy link
Contributor Author

/retest

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Oct 16, 2023
@ti-chi-bot ti-chi-bot bot added the lgtm label Oct 17, 2023
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Oct 17, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: asddongmen, nongfushanquan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added approved and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Oct 17, 2023
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Oct 17, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-10-16 09:48:58.912540603 +0000 UTC m=+1650536.499650747: ☑️ agreed by nongfushanquan.
  • 2023-10-17 01:12:14.053875983 +0000 UTC m=+1705931.640986129: ☑️ agreed by asddongmen.

@3AceShowHand
Copy link
Contributor Author

/test cdc-integration-storage-test

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Oct 17, 2023

@3AceShowHand: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test cdc-integration-kafka-test
  • /test cdc-integration-mysql-test
  • /test cdc-integration-pulsar-test
  • /test cdc-integration-storage-test
  • /test dm-compatibility-test
  • /test dm-integration-test
  • /test engine-integration-test
  • /test verify
  • /test wip-pull-build
  • /test wip-pull-check
  • /test wip-pull-unit-test-cdc
  • /test wip-pull-unit-test-dm
  • /test wip-pull-unit-test-engine

Use /test all to run the following jobs that were automatically triggered:

  • pingcap/tiflow/ghpr_verify
  • pingcap/tiflow/pull_cdc_integration_kafka_test
  • pingcap/tiflow/pull_cdc_integration_storage_test
  • pingcap/tiflow/pull_cdc_integration_test
  • pingcap/tiflow/pull_dm_compatibility_test
  • pingcap/tiflow/pull_dm_integration_test
  • pingcap/tiflow/pull_engine_integration_test

In response to this:

/test cdc-integration-storage-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@3AceShowHand
Copy link
Contributor Author

/test cdc-integration-storage-test

@3AceShowHand
Copy link
Contributor Author

/test dm-integration-test

1 similar comment
@3AceShowHand
Copy link
Contributor Author

/test dm-integration-test

@3AceShowHand
Copy link
Contributor Author

/test cdc-integration-mysql-test

@3AceShowHand
Copy link
Contributor Author

/test cdc-integration-kafka-test

1 similar comment
@3AceShowHand
Copy link
Contributor Author

/test cdc-integration-kafka-test

@3AceShowHand 3AceShowHand added the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Oct 17, 2023
@ti-chi-bot ti-chi-bot bot merged commit 3e78420 into pingcap:master Oct 17, 2023
4 checks passed
@ti-chi-bot ti-chi-bot mentioned this pull request Oct 17, 2023
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.5: #9918.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Oct 17, 2023
@3AceShowHand @ti-chi-bot
kafka(ticdc): support columns dispatcher. (pingcap#9863)
3802006 
close pingcap#9862
ti-chi-bot bot pushed a commit that referenced this pull request Oct 18, 2023
@ti-chi-bot
kafka(ticdc): support columns dispatcher. (#9863) (#9918)
7c9bc3f 
close #9862
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CharlesCheung96 CharlesCheung96 CharlesCheung96 left review comments

@nongfushanquan nongfushanquan nongfushanquan approved these changes

@asddongmen asddongmen asddongmen approved these changes

@sdojjy sdojjy Awaiting requested review from sdojjy

@hicqu hicqu Awaiting requested review from hicqu

Assignees
No one assigned
Labels
approved lgtm needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

kafka sink support columns dispatcher to calculate the partition number
5 participants
@3AceShowHand @ti-chi-bot @asddongmen @CharlesCheung96 @nongfushanquan