Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TEZ-4569: SCATTER_GATHER + BROADCAST hangs on DAG Recovery #361

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

okumin
Copy link
Contributor

@okumin okumin commented Jun 12, 2024

Let an AM correctly restore its state and restart tasks.
https://issues.apache.org/jira/browse/TEZ-4569

@tez-yetus

This comment was marked as outdated.

@tez-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+0 🆗 mvndep 7m 2s Maven dependency ordering for branch
+1 💚 mvninstall 16m 6s master passed
+1 💚 compile 1m 21s master passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚 compile 1m 12s master passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+1 💚 checkstyle 1m 33s master passed
+1 💚 javadoc 1m 4s master passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚 javadoc 0m 44s master passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+0 🆗 spotbugs 0m 48s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 49s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 10s Maven dependency ordering for patch
+1 💚 mvninstall 0m 48s the patch passed
+1 💚 compile 0m 52s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚 javac 0m 52s the patch passed
+1 💚 compile 0m 48s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+1 💚 javac 0m 48s the patch passed
-0 ⚠️ checkstyle 0m 12s tez-tests: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 0m 20s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1
+1 💚 javadoc 0m 20s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
+1 💚 findbugs 1m 59s the patch passed
_ Other Tests _
+1 💚 unit 5m 11s tez-dag in the patch passed.
+1 💚 unit 41m 25s tez-tests in the patch passed.
+1 💚 asflicense 0m 31s The patch does not generate ASF License warnings.
86m 47s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/2/artifact/out/Dockerfile
GITHUB PR #361
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname Linux 1a7a0c602b4f 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/tez.sh
git revision master / e08d027
Default Java Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~22.04.1-b08
checkstyle https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/2/artifact/out/diff-checkstyle-tez-tests.txt
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/2/testReport/
Max. process+thread count 1162 (vs. ulimit of 5500)
modules C: tez-dag tez-tests U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-361/2/console
versions git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@okumin okumin changed the title [WIP] TEZ-4569: SCATTER_GATHER + BROADCAST hangs on DAG Recovery TEZ-4569: SCATTER_GATHER + BROADCAST hangs on DAG Recovery Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants