Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sink(ticdc): limit encoder-concurrency to avoid crash #11775

Merged
merged 8 commits into from
Nov 25, 2024

Conversation

wk989898
Copy link
Collaborator

@wk989898 wk989898 commented Nov 19, 2024

What problem does this PR solve?

Issue Number: close #11773

What is changed and how it works?

As title. The maximum value is CPU count * 10

Check List

Tests

  • Unit test
  • Integration test

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

`None`.

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-triage-completed release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Nov 19, 2024
Copy link

codecov bot commented Nov 19, 2024

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 55.1680%. Comparing base (ea35677) to head (5961029).

Additional details and impacted files
Components Coverage Δ
cdc 59.6172% <0.0000%> (+0.0328%) ⬆️
dm 50.0384% <ø> (+0.0019%) ⬆️
engine 53.2054% <ø> (-0.0339%) ⬇️
Flag Coverage Δ
unit 55.1680% <0.0000%> (+0.0126%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #11775        +/-   ##
================================================
+ Coverage   55.1554%   55.1680%   +0.0126%     
================================================
  Files          1002       1002                
  Lines        137399     137402         +3     
================================================
+ Hits          75783      75802        +19     
+ Misses        56073      56059        -14     
+ Partials       5543       5541         -2     
---- 🚨 Try these New Features:

@wk989898 wk989898 changed the title sink(ticdc): limit encoder-concurrency to avoid OOM sink(ticdc): limit encoder-concurrency to avoid crash Nov 21, 2024
@wk989898 wk989898 added affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. labels Nov 22, 2024
@wk989898
Copy link
Collaborator Author

/retest

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Nov 22, 2024
@wk989898 wk989898 added the affects-6.1 This bug affects the 6.1.x(LTS) versions. label Nov 22, 2024
@wk989898
Copy link
Collaborator Author

/retest

1 similar comment
@wk989898
Copy link
Collaborator Author

/retest

@wk989898
Copy link
Collaborator Author

/run-check-issue-triage-complete

@3AceShowHand
Copy link
Contributor

/retest

@wk989898
Copy link
Collaborator Author

/cherry-pick release-8.5

@ti-chi-bot
Copy link
Member

@wk989898: once the present PR merges, I will cherry-pick it on top of release-8.5 in the new PR and assign it to you.

In response to this:

/cherry-pick release-8.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@wk989898 wk989898 removed affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. labels Nov 25, 2024
@ti-chi-bot ti-chi-bot bot merged commit 3258043 into pingcap:master Nov 25, 2024
26 checks passed
@ti-chi-bot
Copy link
Member

@wk989898: new pull request created to branch release-8.5: #11792.

In response to this:

/cherry-pick release-8.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

@wk989898: cannot checkout release-8.1 release-7.5 release-7.1 release-6.5 release-6.1: error checking out release-8.1 release-7.5 release-7.1 release-6.5 release-6.1: exit status 1. output: error: pathspec 'release-8.1 release-7.5 release-7.1 release-6.5 release-6.1' did not match any file(s) known to git

In response to this:

/cherry-pick release-8.1 release-7.5 release-7.1 release-6.5 release-6.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

@wk989898: cannot checkout release-8.1, release-7.5, release-7.1, release-6.5, release-6.1: error checking out release-8.1, release-7.5, release-7.1, release-6.5, release-6.1: exit status 1. output: error: pathspec 'release-8.1, release-7.5, release-7.1, release-6.5, release-6.1' did not match any file(s) known to git

In response to this:

/cherry-pick release-8.1, release-7.5, release-7.1, release-6.5, release-6.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@wk989898
Copy link
Collaborator Author

/cherry-pick release-8.1

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Nov 26, 2024
@ti-chi-bot
Copy link
Member

@wk989898: new pull request created to branch release-8.1: #11795.

In response to this:

/cherry-pick release-8.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@wk989898
Copy link
Collaborator Author

/cherry-pick release-7.5

@wk989898
Copy link
Collaborator Author

/cherry-pick release-7.1

@wk989898
Copy link
Collaborator Author

/cherry-pick release-6.5

@wk989898
Copy link
Collaborator Author

/cherry-pick release-6.1

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Nov 26, 2024
@ti-chi-bot
Copy link
Member

@wk989898: new pull request created to branch release-7.5: #11796.

In response to this:

/cherry-pick release-7.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Nov 26, 2024
@ti-chi-bot
Copy link
Member

@wk989898: new pull request created to branch release-7.1: #11797.

In response to this:

/cherry-pick release-7.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Nov 26, 2024
@ti-chi-bot
Copy link
Member

@wk989898: new pull request created to branch release-6.5: #11798.

In response to this:

/cherry-pick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Nov 26, 2024
@ti-chi-bot
Copy link
Member

@wk989898: new pull request created to branch release-6.1: #11799.

In response to this:

/cherry-pick release-6.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CDC OOM when encoder-concurrency is too large
4 participants