Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Job 1: Failed #3

Open
mflipe opened this issue May 8, 2021 · 0 comments
Open

Job 1: Failed #3

mflipe opened this issue May 8, 2021 · 0 comments

Comments

@mflipe
Copy link

mflipe commented May 8, 2021

Job 1 Failed

  • Compiled with SBT 1.4 on windows
  • No changes to the source code
  • Executed on Ubuntu WSL 2

LOG:

$ hadoop jar hw2_dblp_mapred.jar 1 input
2021-05-07 18:11:03,386 INFO hw2.RunJobs$: Starting up MapReduce job..
2021-05-07 18:11:03,500 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /0.0.0.0:8032
2021-05-07 18:11:04,078 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2021-05-07 18:11:04,161 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/hduser/.staging/job_1620420770486_0001
2021-05-07 18:11:06,687 INFO input.FileInputFormat: Total input files to process : 1
2021-05-07 18:11:07,251 INFO mapreduce.JobSubmitter: number of splits:24
2021-05-07 18:11:08,085 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1620420770486_0001
2021-05-07 18:11:08,085 INFO mapreduce.JobSubmitter: Executing with tokens: []
2021-05-07 18:11:08,245 INFO conf.Configuration: resource-types.xml not found
2021-05-07 18:11:08,246 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2021-05-07 18:11:08,611 INFO impl.YarnClientImpl: Submitted application application_1620420770486_0001
2021-05-07 18:11:08,640 INFO mapreduce.Job: The url to track the job: http://[MYUSER].localdomain:8088/proxy/application_1620420770486_0001/
2021-05-07 18:11:08,640 INFO mapreduce.Job: Running job: job_1620420770486_0001
2021-05-07 18:11:15,697 INFO mapreduce.Job: Job job_1620420770486_0001 running in uber mode : false
2021-05-07 18:11:15,698 INFO mapreduce.Job:  map 0% reduce 0%
2021-05-07 18:11:30,885 INFO mapreduce.Job: Task Id : attempt_1620420770486_0001_m_000000_0, Status : FAILED
Error: java.util.NoSuchElementException: head of empty list
        at scala.collection.immutable.Nil$.head(List.scala:629)
        at scala.collection.immutable.Nil$.head(List.scala:628)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1(VenueTopTenAuthorsMapper.scala:33)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1$adapted(VenueTopTenAuthorsMapper.scala:28)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:28)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:11)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)

2021-05-07 18:11:45,041 INFO mapreduce.Job:  map 1% reduce 0%
2021-05-07 18:11:58,103 INFO mapreduce.Job:  map 2% reduce 0%
2021-05-07 18:12:15,205 INFO mapreduce.Job:  map 3% reduce 0%
2021-05-07 18:12:27,277 INFO mapreduce.Job:  map 4% reduce 0%
2021-05-07 18:12:39,344 INFO mapreduce.Job:  map 5% reduce 0%
2021-05-07 18:12:53,448 INFO mapreduce.Job:  map 6% reduce 0%
2021-05-07 18:13:09,529 INFO mapreduce.Job:  map 7% reduce 0%
2021-05-07 18:13:21,595 INFO mapreduce.Job:  map 8% reduce 0%
2021-05-07 18:13:33,643 INFO mapreduce.Job:  map 9% reduce 0%
2021-05-07 18:13:44,676 INFO mapreduce.Job: Task Id : attempt_1620420770486_0001_m_000006_0, Status : FAILED
Error: java.util.NoSuchElementException: head of empty list
        at scala.collection.immutable.Nil$.head(List.scala:629)
        at scala.collection.immutable.Nil$.head(List.scala:628)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1(VenueTopTenAuthorsMapper.scala:33)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1$adapted(VenueTopTenAuthorsMapper.scala:28)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:28)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:11)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)

2021-05-07 18:13:45,679 INFO mapreduce.Job:  map 8% reduce 0%
2021-05-07 18:13:51,708 INFO mapreduce.Job:  map 9% reduce 0%
2021-05-07 18:13:52,728 INFO mapreduce.Job: Task Id : attempt_1620420770486_0001_m_000000_1, Status : FAILED
Error: java.util.NoSuchElementException: head of empty list
        at scala.collection.immutable.Nil$.head(List.scala:629)
        at scala.collection.immutable.Nil$.head(List.scala:628)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1(VenueTopTenAuthorsMapper.scala:33)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1$adapted(VenueTopTenAuthorsMapper.scala:28)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:28)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:11)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)

2021-05-07 18:14:06,788 INFO mapreduce.Job: Task Id : attempt_1620420770486_0001_m_000005_0, Status : FAILED
Error: java.util.NoSuchElementException: head of empty list
        at scala.collection.immutable.Nil$.head(List.scala:629)
        at scala.collection.immutable.Nil$.head(List.scala:628)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1(VenueTopTenAuthorsMapper.scala:33)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1$adapted(VenueTopTenAuthorsMapper.scala:28)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:28)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:11)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)

2021-05-07 18:14:07,794 INFO mapreduce.Job:  map 7% reduce 0%
2021-05-07 18:14:09,801 INFO mapreduce.Job:  map 8% reduce 0%
2021-05-07 18:14:15,820 INFO mapreduce.Job: Task Id : attempt_1620420770486_0001_m_000000_2, Status : FAILED
Error: java.util.NoSuchElementException: head of empty list
        at scala.collection.immutable.Nil$.head(List.scala:629)
        at scala.collection.immutable.Nil$.head(List.scala:628)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1(VenueTopTenAuthorsMapper.scala:33)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.$anonfun$map$1$adapted(VenueTopTenAuthorsMapper.scala:28)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:28)
        at com.samujjwaal.hw2.mappers.VenueTopTenAuthorsMapper.map(VenueTopTenAuthorsMapper.scala:11)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)

2021-05-07 18:14:27,889 INFO mapreduce.Job:  map 9% reduce 0%
2021-05-07 18:14:39,927 INFO mapreduce.Job:  map 10% reduce 0%
2021-05-07 18:14:51,963 INFO mapreduce.Job:  map 11% reduce 0%
2021-05-07 18:15:04,008 INFO mapreduce.Job:  map 12% reduce 0%
2021-05-07 18:15:12,038 INFO mapreduce.Job:  map 14% reduce 0%
2021-05-07 18:15:15,053 INFO mapreduce.Job:  map 17% reduce 0%
2021-05-07 18:15:20,068 INFO mapreduce.Job:  map 100% reduce 100%
2021-05-07 18:15:22,075 INFO mapreduce.Job: Job job_1620420770486_0001 failed with state FAILED due to: Task failed task_1620420770486_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0

2021-05-07 18:15:22,130 INFO mapreduce.Job: Counters: 39
        File System Counters
                FILE: Number of bytes read=0
                FILE: Number of bytes written=137105630
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=536871973
                HDFS: Number of bytes written=0
                HDFS: Number of read operations=12
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=0
                HDFS: Number of bytes read erasure-coded=0
        Job Counters
                Failed map tasks=6
                Killed map tasks=19
                Killed reduce tasks=1
                Launched map tasks=14
                Other local map tasks=5
                Data-local map tasks=9
                Total time spent by all maps in occupied slots (ms)=1445398
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=1445398
                Total vcore-milliseconds taken by all map tasks=1445398
                Total megabyte-milliseconds taken by all map tasks=1480087552
        Map-Reduce Framework
                Map input records=1030940
                Map output records=3175471
                Map output bytes=129695240
                Map output materialized bytes=136046206
                Input split bytes=452
                Combine input records=0
                Spilled Records=3175471
                Failed Shuffles=0
                Merged Map outputs=0
                GC time elapsed (ms)=10286
                CPU time spent (ms)=907880
                Physical memory (bytes) snapshot=2259881984
                Virtual memory (bytes) snapshot=10277044224
                Total committed heap usage (bytes)=1890058240
                Peak Map Physical memory (bytes)=586129408
                Peak Map Virtual memory (bytes)=2577076224
        File Input Format Counters
                Bytes Read=536871521
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
DBLP MapReduce
Status: To do
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mflipe