Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] when submitting tasks in batches, em resources may exceed the threshold #4917

Closed
wants to merge 1 commit into from
Closed

[BUG] when submitting tasks in batches, em resources may exceed the threshold #4917

wants to merge 1 commit into from

Conversation

sjgllgh
Copy link
Contributor

@sjgllgh sjgllgh commented Sep 26, 2023

What is the purpose of the change

Fix bug: when submitting tasks in batches, em resources may exceed the threshold.

Related issues/PRs

Related issues: #4916

Brief change log

  • Modify the DefaultResourceManager.java to fix the bug.

Checklist

  • I have read the Contributing Guidelines on pull requests.
  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the Linkis mailing list first)
  • If this is a code change: I have written unit tests to fully verify the new behavior.

@peacewong
Copy link
Contributor

The logical judgment here is considered as follows:

  1. The ECM tag is not locked first for resource judgment. If the resource is judged, it will continue. Therefore, there is a certain probability of negative resources, but there will not be too many situations.
  2. The reason for this consideration is to reduce lock waiting, because here is a concurrent lock grab, so the lock granularity should be as small as possible to reduce concurrency pressure.
  3. If the usercreator lock is placed at a higher level, it will cause more lock waits.

这里的逻辑判断是这样考虑的:

  1. ECM标签先不锁进行资源判断如果资源判断过就继续,所以存在一定概率出现负资源,但是不会出现太多情况
  2. 之所以这样考虑是为了减少锁的等待,因为这里是并发抢锁的,所以锁的粒度尽量小减少并发压力
  3. 如果把usercreator的锁放到更上一层,会导致更多的锁等待

@sjgllgh
Copy link
Contributor Author

sjgllgh commented Sep 27, 2023

Thank you for your answer. Different scenarios may have different considerations.

The logical judgment here is considered as follows:

  1. The ECM tag is not locked first for resource judgment. If the resource is judged, it will continue. Therefore, there is a certain probability of negative resources, but there will not be too many situations.
  2. The reason for this consideration is to reduce lock waiting, because here is a concurrent lock grab, so the lock granularity should be as small as possible to reduce concurrency pressure.
  3. If the usercreator lock is placed at a higher level, it will cause more lock waits.

这里的逻辑判断是这样考虑的:

  1. ECM标签先不锁进行资源判断如果资源判断过就继续,所以存在一定概率出现负资源,但是不会出现太多情况
  2. 之所以这样考虑是为了减少锁的等待,因为这里是并发抢锁的,所以锁的粒度尽量小减少并发压力
  3. 如果把usercreator的锁放到更上一层,会导致更多的锁等待

@sjgllgh sjgllgh closed this Sep 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants