forked from OpenPOWER-BigData/HDP2.5-tez
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGES.txt
1245 lines (1208 loc) · 86.5 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Apache Tez Change Log
=====================
Release 0.7.0: 2015-05-18
INCOMPATIBLE CHANGES
TEZ-2176. Move all logging to slf4j. (commons-logging jar no longer part of Tez tar)
TEZ-1993. Implement a pluggable InputSizeEstimator for grouping fairly.
TEZ-2424. Bump up max counter group name length limit to account for per_io counters.
Default max limit increased. Should not affect existing users.
ALL CHANGES:
TEZ-2446. Tez UI: Add tezVersion details when downloading timeline data for offline use
TEZ-2432. Syntax error in DOAP file release section
TEZ-2445. Disable the object cleanup in local mode in LogicalIOProcessorRuntimeTask.
TEZ-2057. tez-dag/pom.xml contains versions for dependencies.
TEZ-2435. Add public key to KEYS
TEZ-2421. Deadlock in AM because attempt and vertex locking each other out
TEZ-2426. Ensure the eventRouter thread completes before switching to a new task and thread safety fixes in IPOContexts.
TEZ-2412. Should kill vertex in DAGImpl#VertexRerunWhileCommitting
TEZ-2410. VertexGroupCommitFinishedEvent & VertexCommitStartedEvent is not logged correctly
TEZ-776. Reduce AM mem usage caused by storing TezEvents
TEZ-2423. Tez UI: Remove Attempt Index column from task->attempts page
TEZ-2416. Tez UI: Make tooltips display faster.
TEZ-2404. Handle DataMovementEvent before its TaskAttemptCompletedEvent
TEZ-2424. Bump up max counter group name length limit to account for per_io counters.
TEZ-2417. Tez UI: Counters are blank in the Attempts page if all attempts failed
TEZ-2366. Pig tez MiniTezCluster unit tests fail intermittently after TEZ-2333
TEZ-2406. Tez UI: Display per-io counter columns in task and attempt pages under vertex
TEZ-2384. Add warning message in the case of prewarn under non-session mode.
TEZ-2415. PMC RDF needs to use asfext:pmc, not asfext:PMC.
TEZ-2401. Tez UI: All-dag page has duration keep counting for KILLED dag.
TEZ-2392. Have all readers throw an Exception on incorrect next() usage.
TEZ-2408. TestTaskAttempt fails to compile against hadoop-2.4 and hadoop-2.2.
TEZ-2405. PipelinedSorter can throw NPE with custom compartor.
TEZ-1897. Create a concurrent version of AsyncDispatcher
TEZ-2394. Issues when there is an error in VertexManager callbacks
TEZ-2386. Tez UI: Inconsistent usage of icon colors
TEZ-2395. Tez UI: Minimum/Maximum Duration show a empty bracket next to 0 secs when you purposefully failed a job.
TEZ-2360. per-io counters flag should generate both overall and per-edge counters
TEZ-2389. Tez UI: Sort by attempt-no is incorrect in attempts pages.
TEZ-2363: Fix off-by-one error in REDUCE_INPUT_RECORDS counter
TEZ-2383. Cleanup input/output/processor contexts in LogicalIOProcessorRuntimeTask.
TEZ-2084. Tez UI: Stacktrace format info is lost in diagnostics
TEZ-2374. Fix build break against hadoop-2.2 due to TEZ-2325.
TEZ-2314. Tez task attempt failures due to bad event serialization
TEZ-2368. Make a dag identifier available in Context classes.
TEZ-2325. Route status update event directly to the attempt.
TEZ-2358. Pipelined Shuffle: MergeManager assumptions about 1 merge per source-task.
TEZ-2342. TestFaultTolerance.testRandomFailingTasks fails due to timeout.
TEZ-2362. State Change Notifier Thread should be stopped when dag is
completed
TEZ-2364. Resolve config parameter replacement on the client, before sending them to the AM.
TEZ-2298. Avoid logging full exception trace in TaskRunner when it's not the main error reason and is ignored.
TEZ-2248. VertexImpl/DAGImpl.checkForCompletion have too many termination cause checks
TEZ-2341. TestMockDAGAppMaster.testBasicCounters fails on windows
TEZ-2352. Move getTaskStatistics into the RuntimeTask class.
TEZ-2357. Tez UI: misc.js.orig is committed by accident
TEZ-2261. Should add diagnostics in DAGAppMaster when recovery error happens
TEZ-2340. TestRecoveryParser fails
TEZ-2345. Tez UI: Enable cell level loading in all DAGs table
TEZ-2330. Create reconfigureVertex() API for input based initialization
TEZ-2292. Add e2e test for error reporting when vertex manager invokes
plugin APIs
TEZ-2308. Add set/get of record counts in task/vertex statistics
TEZ-2344. Tez UI: Equip basic-ember-table's cell level loading for all use cases in all DAGs table
TEZ-2313. Regression in handling obsolete events in ShuffleScheduler.
TEZ-2212. Notify components on DAG completion.
TEZ-2328. Add tez.runtime.sorter.class & rename tez.runtime.sort.threads
to tez.runtime.pipelined.sorter.sort.threads.
TEZ-2333. Enable local fetch optimization by default.
TEZ-2310. Deadlock caused by StateChangeNotifier sending notifications on
thread holding locks
TEZ-1969. Stop the DAGAppMaster when a local mode client is stopped
TEZ-714. OutputCommitters should not run in the main AM dispatcher thread
TEZ-2323. Fix TestOrderedWordcount to use MR memory configs.
TEZ-1482. Fix memory issues for Local Mode running concurrent tasks
TEZ-2033. Update TestOrderedWordCount to add processor configs as history text
and use MR configs correctly
TEZ-2318. Tez UI: source and sink page is broken as they are not populated.
TEZ-2016. Tez UI: Dag View Fit and Finish
TEZ-2252. Tez UI: in graphical view some of the sinks are hidden as they overlap
TEZ-2275. Tez UI: enable faster loading and caching of data in tables
TEZ-2234. Add API for statistics information - allow vertex managers to get
output size per source vertex
TEZ-2274. Tez UI: full data loading, client side search and sort for other pages
TEZ-2301. Switch Tez Pre-commit builds to use tezqa user.
TEZ-2299. Invalid dag creation in MRRSleepJob post TEZ-2293.
TEZ-2290. Scale memory for Default Sorter down to a max of 2047 MB if configured higher.
TEZ-2233. Allow EdgeProperty of an edge to be changed by VertexManager
TEZ-2293. When running in "mr" mode, always use MR config settings.
TEZ-2273. Tez UI: Support client side searching & sorting for dag tasks page
TEZ-2223. TestMockDAGAppMaster fails due to TEZ-2210 on mac.
TEZ-2236. Tez UI: Support loading of all tasks in the dag tasks page
TEZ-2159. Tez UI: download timeline data for offline use.
TEZ-2269. DAGAppMaster becomes unresponsive (post TEZ-2149).
TEZ-2243. documentation should explicitly specify protobuf 2.5.0.
TEZ-2232. Allow setParallelism to be called multiple times before tasks get
scheduled
TEZ-2265. All inputs/outputs in a task share the same counter object
TEZ-2251. Race condition in VertexImpl & Edge causes DAG to hang.
TEZ-2264. Remove unused taskUmbilical reference in TezTaskRunner, register as running late.
TEZ-2149. Optimizations for the timed version of DAGClient.getStatus.
TEZ-2213. For the ordered case, enabling pipelined shuffle should automatically disable final merge.
TEZ-2204. TestAMRecovery increasingly flaky on jenkins builds.
TEZ-2209. Fix pipelined shuffle to fetch data from any one attempt
TEZ-2210. Record DAG AM CPU usage stats
TEZ-2203. Intern strings in tez counters
TEZ-2202. Fix LocalTaskExecutionThread ID to the standard thread numbering.
TEZ-2059. Remove TaskEventHandler in TestDAGImpl
TEZ-2191. Simulation improvements to MockDAGAppMaster
TEZ-2195. TestTezJobs::testInvalidQueueSubmission/testInvalidQueueSubmissionToSession
fail with hadoop branch-2.
TEZ-1827. MiniTezCluster takes 10 minutes to shut down.
TEZ-2178. YARN-3122 breaks tez compilation with hadoop 2.7.0.
TEZ-2174. Make task priority available to TaskAttemptListener.
TEZ-2169. Add NDC context to various threads and pools.
TEZ-2171. Remove unused metrics code.
TEZ-2001. Support pipelined data transfer for ordered output.
TEZ-2170. Incorrect its in README.md.
TEZ-2070. Controller class of output should be committer rather than initializer in DAG's dot file.
TEZ-2083. Make PipelinedSorter as the default sorter.
TEZ-1967. Add a monitoring API on DAGClient which returns after a time interval or on DAG final state change.
TEZ-2130. Send the sessionToken as part of the AM CLC.
TEZ-1935. Organization should be removed from http://tez.apache.org/team-list.html.
TEZ-2009. Change license/copyright headers to 2015.
TEZ-2085. PipelinedSorter should bail out (on BufferOverflowException) instead of retrying continuously.
TEZ-167. Create tests for MR Combiner.
TEZ-2080. Localclient should be using tezconf in init instead of yarnconf.
TEZ-2072. Add missing Private annotation to createDAG in the DAG API class.
TEZ-2095. master branch fails to compile against hadoop-2.4.
TEZ-2093. Add events to MockDAGAppMaster and add e2e test for event routing
TEZ-2075. Incompatible issue caused by TEZ-1233 that TezConfiguration.TEZ_SITE_XML is made private
TEZ-2082. Race condition in TaskAttemptListenerImpTezDag.getTask()
TEZ-1233. Allow configuration of framework parameters per vertex.
TEZ-2045. TaskAttemptListener should not pull Tasks from AMContainer. Instead these should be registered with the listener.
TEZ-1914. VertexManager logic should not run on the central dispatcher
TEZ-2023. Refactor logIndividualFetchComplete() to be common for both shuffle-schedulers.
TEZ-1999. IndexOutOfBoundsException during merge.
TEZ-2000. Source vertex exists error during DAG submission.
TEZ-2008. Add methods to SecureShuffleUtils to verify a reply based on a provided Key.
TEZ-1995. Build failure against hadoop 2.2.
TEZ-1997. Remove synchronization DefaultSorter::isRLENeeded() (Causes sorter to hang indefinitely in large jobs).
TEZ-1996. Update Website after 0.6.0
TEZ-1803. Support > 2gb sort buffer in pipelinedsorter.
TEZ-1826. Add option to disable split grouping and local mode option for tez-examples.
TEZ-1982. TezChild setupUgi should not be using environment.
TEZ-1980. Suppress tez-dag findbugs warnings until addressed.
TEZ-1855. Avoid scanning for previously written files within Inputs / Outputs.
TEZ-1902. Fix findbugs warnings in tez-mapreduce.
TEZ-1963. Fix post memory merge to be > 2 GB.
TEZ-1901. Fix findbugs warnings in tez-examples.
TEZ-1941. Memory provided by *Context.getAvailableMemory needs to be setup explicitly.
TEZ-1879. Create local UGI instances for each task and the AM, when running in LocalMode.
TEZ-1661. LocalTaskScheduler hangs when shutdown.
TEZ-1951. Fix general findbugs warnings in tez-dag.
TEZ-1905. Fix findbugs warnings in tez-tests.
TEZ-1945. Remove 2 GB memlimit restriction in MergeManager.
TEZ-1913. Reduce deserialize cost in ValuesIterator.
TEZ-1917. Examples should extend TezExampleBase.
TEZ-1892. Add hashCode and equals for Vertex/VertexGroup/Edge/GroupInputEdge.
TEZ-1904. Fix findbugs warnings in tez-runtime-library module.
TEZ-1903. Fix findbugs warnings in tez-runtime-internal module.
TEZ-1896. Move the default heartbeat timeout and checkinterval to TezConfiguration.
TEZ-1274. Remove Key/Value type checks in IFile.
TEZ-1912. Merge exceptions are thrown when enabling tez.runtime.shuffle.memory-to-memory.enable && tez.runtime.shuffle.memory-to-memory.segments.
TEZ-1922. Fix comments: add UNSORTED_OUTPUT to TEZ_TASK_SCALE_MEMORY_WEIGHTED_RATIOS.
TEZ-485. Get rid of TezTaskStatus.
TEZ-1899. Fix findbugs warnings in tez-common module.
TEZ-1898. Fix findbugs warnings in tez-api module.
TEZ-1906. Fix findbugs warnings in tez-yarn-timeline-history-with-acls.
TEZ-1767. Enable RLE in reducer side merge codepath.
TEZ-1837. Restrict usage of Environment variables to main methods.
TEZ-1867. Create new central dispatcher for Tez AM
TEZ-1844. Shouldn't invoke system.exit in local mode when AM is failed to start.
TEZ-1889. Fix test-patch to provide correct findbugs report.
TEZ-1313. Setup pre-commit build to test submitted patches.
TEZ-1856. Remove LocalOnFileSortedOutput, LocalMergedInput, LocalTaskOutputFiles.
TEZ-1949. Whitelist TEZ_RUNTIME_OPTIMIZE_SHARED_FETCH for broadcast edges.
TEZ-1593. Refactor PipelinedSorter to remove all MMAP based ByteBuffer references.
Release 0.6.2: Unreleased
INCOMPATIBLE CHANGES
ALL CHANGES:
TEZ-2548. TezClient submitDAG can hang if the AM is in the process of shutting down.
TEZ-2534. Error handling summary event when shutting down AM.
TEZ-2511. Add exitCode to diagnostics when container fails.
TEZ-2489. Disable warn log for Timeline ACL error when tez.allow.disabled.timeline-domains set to true.
TEZ-2509. YarnTaskSchedulerService should not try to allocate containers if AM is shutting down.
TEZ-2483. Tez should close task if processor fail
Release 0.6.1: 2015-05-18
INCOMPATIBLE CHANGES
ALL CHANGES:
TEZ-2437. FilterLinesByWord NPEs when run in Localmode
TEZ-2057. tez-dag/pom.xml contains versions for dependencies.
TEZ-2282. Delimit reused yarn container logs (stderr, stdout, syslog) with task attempt start/stop events
TEZ-2396. pig-tez-tfile-parser pom is hard coded to depend on 0.6.0-SNAPSHOT version.
TEZ-2237. Valid events should be sent out when an Output is not started.
TEZ-1988. Tez UI: does not work when using file:// in a browser
TEZ-2390. tez-tools swimlane tool fails to parse large jobs >8K containers
TEZ-2256. Avoid use of BufferTooSmallException to signal end of buffer in UnorderedPartitionedKVWriter
TEZ-2380. Disable fall back to reading from timeline if timeline disabled.
TEZ-2226. Disable writing history to timeline if domain creation fails.
TEZ-2259. Push additional data to Timeline for Recovery for better consumption in UI.
TEZ-2365. Update tez-ui war's license/notice to reflect OFL license correctly.
TEZ-2329. UI Query on final dag status performance improvement
TEZ-2287. Deprecate VertexManagerPluginContext.getTaskContainer().
TEZ-1909. Remove need to copy over all events from attempt 1 to attempt 2 dir
TEZ-2061. Tez UI: vertex id column and filter on tasks page should be changed to vertex name
TEZ-2242. Refactor ShuffleVertexManager code
TEZ-2205. Tez still tries to post to ATS when yarn.timeline-service.enabled=false.
TEZ-2047. Build fails against hadoop-2.2 post TEZ-2018
TEZ-2064. SessionNotRunning Exception not thrown is all cases
TEZ-2189. Tez UI live AM tracking url only works for localhost addresses
TEZ-2179. Timeline relatedentries missing cause exaggerated warning.
TEZ-2168. Fix application dependencies on mutually exclusive artifacts: tez-yarn-timeline-history
and tez-yarn-timeline-history-with-acls.
TEZ-2190. TestOrderedWordCount fails when generateSplitsInClient set to true.
TEZ-2091. Add support for hosting TEZ_UI with nodejs.
TEZ-2165. Tez UI: DAG shows running status if killed by RM in some cases.
TEZ-2158. TEZ UI: Display dag/vertex names, and task/attempt index in breadcrumb.
TEZ-2160. Tez UI: App tracking URL should support navigation back.
TEZ-2147. Swimlanes: Improved tooltip
TEZ-2142. TEZ UI: Breadcrumb border color looks out of place in wrapped mode.
TEZ-2134. TEZ UI: On request failure, display request URL and server name in error bar.
TEZ-2136. Some enhancements to the new Tez UI.
TEZ-2135. ACL checks handled incorrectly in AMWebController.
TEZ-1990. Tez UI: DAG details page shows Nan for end time when a DAG is running.
TEZ-2116. Tez UI: dags page filter does not work if more than one filter is specified.
TEZ-2106. TEZ UI: Display data load time, and add a refresh button for items that can be refreshed.
TEZ-2114. Tez UI: task/task attempt status is not available when its running.
TEZ-2112. Tez UI: fix offset calculation, add home button to breadcrumbs.
TEZ-2038. TEZ-UI DAG is always running in tez-ui when the app is failed but no DAGFinishedEvent is logged.
TEZ-2102. Tez UI: DAG view has hidden edges, dragging DAG by holding vertex causes unintended click.
TEZ-2101. Tez UI: Issues on displaying a table.
TEZ-2092. Tez UI history url handler injects spurious trailing slash.
TEZ-2098. Tez UI: Dag details should be the default page for dag, fix invalid time entries for failed Vertices.
TEZ-2024. TaskFinishedEvent may not be logged in recovery.
TEZ-2031. Tez UI: horizontal scrollbars do not appear in tables, causing them to look truncated.
TEZ-2073. SimpleHistoryLoggingService cannot be read by log aggregation (umask)
TEZ-2078. Tez UI: Task logs url use in-progress url causing various errors.
TEZ-2077. Tez UI: No diagnostics on Task Attempt Details page if task attempt failed.
TEZ-2065. Setting up tez.tez-ui.history-url.base with a trailing slash can result in failures to redirect correctly.
TEZ-2068. Tez UI: Dag view should use full window height, disable webuiservice in localmode.
TEZ-2079. Tez UI: trailing slash in timelineBaseUrl in ui should be handled.
TEZ-2069. Tez UI: appId should link to application in dag details view.
TEZ-2063. Tez UI: Flaky log url in tasks table.
TEZ-2062. Tez UI: Showing 50 elements not working properly.
TEZ-2056. Tez UI: fix VertexID filter,show only tez configs by default,fix appattemptid.
TEZ-2052. Tez UI: log view fixes, show version from build, better handling of ats url config.
TEZ-2043. Tez UI: add progress info from am webservice to dag and vertex views.
TEZ-2032. Update CHANGES.txt to show 0.6.0 is released
TEZ-2018. App Tracking and History URL should point to the Tez UI.
TEZ-2035. Make timeline server putDomain exceptions non-fatal - work-around
TEZ-1929. pre-empted tasks should be marked as killed instead of failed
TEZ-2017. TEZ UI - Dag view throwing error whild re-displaying additionals in some dags.
TEZ-2013. TEZ UI - App Details Page UI Nits
TEZ-2014. Tez UI: Nits : All tables, Vertices Page UI.
TEZ-2012. TEZ UI: Show page number in all tables, and display more readable task/attempt ids.
TEZ-1973. Dag View
TEZ-2010. History payload generated from conf has ${var} placeholders.
TEZ-1946. Tez UI: add source & sink views, add counters to vertices/all task views.
TEZ-1987. Tez UI non-standalone mode uses invalid protocol.
TEZ-1983. Tez UI swimlane task attempt link is broken
Release 0.6.0: 2015-01-23
INCOMPATIBLE CHANGES
ALL CHANGES:
TEZ-1977. Fixup CHANGES.txt with Tez UI jiras
TEZ-1743. Add versions-maven-plugins artifacts to gitignore
TEZ-1968. Tez UI - All vertices of DAG are not listed in vertices page
TEZ-1890. tez-ui web.tar.gz also being uploaded to maven repository
TEZ-1938. Build warning duplicate jersey-json definitions
TEZ-1910. Build fails against hadoop-2.2.0.
TEZ-1882. Tez UI build does not work on Windows
TEZ-1915. Add public key to KEYS
TEZ-1907. Fix javadoc warnings in tez codebase
TEZ-1891. Incorrect number of Javadoc warnings reported
TEZ-1762. Lots of unit tests do not have timeout parameter set.
TEZ-1886. remove deprecation warnings for tez-ui on the console.
TEZ-1875. dropdown filters do not work on vertices and task attempts page.
TEZ-1873. TestTezAMRMClient fails due to host resolution timing out.
TEZ-1881. Setup initial test-patch script for TEZ-1313.
TEZ-1864. move initialization code dependent on config params to App.ready.
TEZ-1870. Time displayed in the UI is in GMT.
TEZ-1858. Docs for deploying/using the Tez UI.
TEZ-1859. TestGroupedSplits has commented out test: testGzip.
TEZ-1868. Document how to do Windows builds due to with ACL symlink build changes.
TEZ-1872. docs/src/site/custom/project-info-report.properties needs license header.
TEZ-1850. Enable deploy for tez-ui war.
TEZ-1841. Remove range versions for dependencies in tez-ui.
TEZ-1854. Failing tests due to host resolution timing out.
TEZ-1860. mvn apache-rat:check broken for tez-ui.
TEZ-1866. remove the "original" directory under tez-ui
TEZ-1591. Add multiDAG session test and move TestLocalMode to tez-tests
TEZ-1769. ContainerCompletedWhileRunningTransition should inherit from TerminatedWhileRunningTransition
TEZ-1849. Fix tez-ui war file licensing.
TEZ-1840. Document TezTaskOutput.
TEZ-1576. Class level comment in {{MiniTezCluster}} ends abruptly.
TEZ-1838. tez-ui/src/main/webapp/bower.json gets updated after compiling source code.
TEZ-1789. Move speculator processing off the central dispatcher.
TEZ-1610. Add additional task counters for fetchers, merger.
TEZ-1847. Fix package name for MiniTezClusterWithTimeline.
TEZ-1846. Build fails with package org.apache.tez.dag.history.logging.ats does not exist.
TEZ-1696. Make Tez use the domain-based timeline ACLs.
TEZ-1835. TestFaultTolerance#testRandomFailingTasks is timing out
TEZ-1832. TestSecureShuffle fails with NoClassDefFoundError: org/bouncycastle/x509/X509V1CertificateGenerator
TEZ-1672. Update jetty to use stable 7.x version - 7.6.16.v20140903.
TEZ-1822. Docs for Timeline/ACLs/HistoryText.
TEZ-1252. Change wording on http://tez.apache.org/team-list.html related to member confusion.
TEZ-1805. Tez client DAG cycle detection should detect self loops
TEZ-1816. It is possible to receive START event when DAG is failed
TEZ-1787. Counters for speculation
TEZ-1773. Add attempt failure cause enum to the attempt failed/killed
history record
TEZ-14. Support MR like speculation capabilities based on latency deviation
from the mean
TEZ-1733. TezMerger should sort FileChunks on size when merging
TEZ-1738. Tez tfile parser for log parsing
TEZ-1627. Remove OUTPUT_CONSUMABLE and related Event in TaskAttemptImpl
TEZ-1736. Add support for Inputs/Outputs in runtime-library to generate history text data.
TEZ-1721. Update INSTALL instructions for clarifying tez client jars
compatibility with runtime tarball on HDFS.
TEZ-1690. TestMultiMRInput tests fail because of user collisions.
TEZ-1687. Use logIdentifier of Vertex for logging.
TEZ-1737. Should add taskNum in VertexFinishedEvent.
TEZ-1772. Failing tests post TEZ-1737.
TEZ-1785. Remove unused snappy-java dependency.
TEZ-1685. Remove YARNMaster which is never used.
TEZ-1797. Create necessary content for Tez DOAP file.
TEZ-1650. Please create a DOAP file for your TLP.
TEZ-1697. DAG submission fails if a local resource added is already part of tez.lib.uris
TEZ-1060 Add randomness to fault tolerance tests
TEZ-1790. DeallocationTaskRequest may been handled before corresponding AllocationTaskRequest in local mode
TEZ-UI CHANGES (TEZ-8):
TEZ-1823. default ATS url should be the same host as ui
TEZ-1817. Add configuration and build details to README
TEZ-1813. display loading and other error messages in tez-ui
TEZ-1809. Provide a error bar to report errors to users in Tez-UI
TEZ-1810. Do not deploy tez-ui war to maven repo
TEZ-1709. Bunch of files in tez-ui missing Apache license header
TEZ-1784. Attempt details in tasks table.
TEZ-1801. Update build instructions for tez-ui
TEZ-1757. Column selector for tables.
TEZ-1794. Vertex view needs a task attempt rollup
TEZ-1791. breadcrumbs for moving between pages.
TEZ-1781. Configurations view ~ New design
TEZ-1605. Landing page for Tez UI
TEZ-1600. Swimlanes View for Tez UI
TEZ-1799. Enable Cross Origin Support in Tez UI
TEZ-1634. Fix compressed IFile shuffle errors
TEZ-1615. Skeleton framework for Tez UI
TEZ-1604. Task View for Tez UI
TEZ-1603. Vertex View for Tez UI.
TEZ-1720. Allow filters in all tables and also to pass in filters using url params.
TEZ-1708. Make UI part of TEZ build process.
TEZ-1617. Shim layer for Tez UI for use within Ambari.
TEZ-1741. App view.
TEZ-1751. Log view & download links in task and task attempt view.
TEZ-1753. Queue in dags view.
TEZ-1765. Allow dropdown lists in table filters.
TEZ-1606. Counters View for DAG, Vertex, and Task.
TEZ-1768. follow up jira to address minor issues in Tez-ui.
TEZ-1783. Wrapper in standalone mode.
TEZ-1820. Fix wrong links.
Release 0.5.4: Unreleased
ALL CHANGES:
TEZ-2566. Allow TaskAttemptFinishedEvent without TaskAttemptStartedEvent when it is KILLED/FAILED
TEZ-2475. Fix a potential hang in Tez local mode caused by incorrectly handled interrupts.
TEZ-2533. AM deadlock when shutdown
TEZ-2537. mapreduce.map.env and mapreduce.reduce.env need to fall back to mapred.child.env for compatibility
TEZ-2304. InvalidStateTransitonException TA_SCHEDULE at START_WAIT during recovery
TEZ-2488. Tez AM crashes if a submitted DAG is configured to use invalid resource sizes.
TEZ-2080. Localclient should be using tezconf in init instead of yarnconf.
TEZ-2369. Add a few unit tests for RootInputInitializerManager.
TEZ-2379. org.apache.hadoop.yarn.state.InvalidStateTransitonException:
Invalid event: T_ATTEMPT_KILLED at KILLED.
TEZ-2397. Translation of LocalResources via Tez plan serialization can be lossy.
TEZ-2221. VertexGroup name should be unqiue
TEZ-1521. VertexDataMovementEventsGeneratedEvent may be logged twice in recovery log
TEZ-2348. EOF exception during UnorderedKVReader.next().
TEZ-1560. Invalid state machine handling for V_SOURCE_VERTEX_RECOVERED in recovery.
TEZ-2305. MR compatibility sleep job fails with IOException: Undefined job output-path
TEZ-2303. ConcurrentModificationException while processing recovery.
TEZ-2334. ContainerManagementProtocolProxy modifies IPC timeout conf without making a copy.
TEZ-2317. Event processing backlog can result in task failures for short
tasks
TEZ-2289. ATSHistoryLoggingService can generate ArrayOutOfBoundsException.
TEZ-2257. Fix potential NPEs in TaskReporter.
TEZ-2192. Relocalization does not check for source.
TEZ-2224. EventQueue empty doesn't mean events are consumed in RecoveryService
TEZ-2240. Fix toUpperCase/toLowerCase to use Locale.ENGLISH.
TEZ-2238. TestContainerReuse flaky
TEZ-2217. The min-held-containers being released prematurely
TEZ-2214. FetcherOrderedGrouped can get stuck indefinitely when MergeManager misses memToDiskMerging
TEZ-1923. FetcherOrderedGrouped gets into infinite loop due to memory pressure
TEZ-2219. Should verify the input_name/output_name to be unique per vertex
TEZ-2186. OOM with a simple scatter gather job with re-use
TEZ-2220. TestTezJobs compile failure in branch 0.5.
TEZ-2199. updateLocalResourcesForInputSplits assumes wrongly that split data is on same FS as the default FS.
TEZ-2162. org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat is not recognized
TEZ-2193. Check returned value from EdgeManagerPlugin before using it
TEZ-2133. Secured Impersonation: Failed to delete tez scratch data dir
TEZ-2058. Flaky test: TestTezJobs::testInvalidQueueSubmission.
TEZ-2037. Should log TaskAttemptFinishedEvent if taskattempt is recovered to KILLED.
TEZ-2071. TestAMRecovery should set test names for test DAGs.
TEZ-1928. Tez local mode hang in Pig tez local mode.
TEZ-1893. Verify invalid -1 parallelism in DAG.verify().
TEZ-900. Confusing message for incorrect queue for some tez examples.
TEZ-2036. OneToOneEdgeManager should enforce that source and destination
tasks have same number
TEZ-1895. Vertex reRunning should decrease successfulMembers of VertexGroupInfo.
TEZ-2020. For 1-1 edge vertex configured event may be sent incorrectly
TEZ-2015. VertexImpl.doneReconfiguringVertex() should check other criteria
before sending notification
TEZ-2011. InputReadyVertexManager not resilient to updates in parallelism
TEZ-1934. TestAMRecovery may fail due to the execution order is not determined.
TEZ-1642. TestAMRecovery sometimes fail.
TEZ-1931. Publish tez version info to Timeline.
TEZ-1942. Number of tasks show in Tez UI with auto-reduce parallelism is misleading.
TEZ-1962. Fix a thread leak in LocalMode.
TEZ-1924. Tez AM does not register with AM with full FQDN causing jobs
to fail in some environments.
TEZ-1878. Task-specific log level override not working in certain conditions.
TEZ-1775. Allow setting log level per logger.
TEZ-1851. FileSystem counters do not differentiate between different FileSystems.
TEZ-1852. Get examples to work in LocalMode.
TEZ-1861. Fix failing test: TestOnFileSortedOutput.
TEZ-1836. Provide better error messages when tez.runtime.io.sort.mb, spill percentage is incorrectly configured.
TEZ-1800. Integer overflow in ExternalSorter.getInitialMemoryRequirement()
Release 0.5.3: 2014-12-10
ALL CHANGES:
TEZ-1925. Remove npm WARN messages from the Tez UI build process.
TEZ-1758. TezClient should provide YARN diagnostics when the AM crashes
TEZ-1742. Improve response time of internal preemption
TEZ-1745. TestATSHistoryLoggingService::testATSHistoryLoggingServiceShutdown can be flaky.
TEZ-1747. Increase test timeout for TestSecureShuffle.
TEZ-1746. Flaky test in TestVertexImpl and TestExceptionPropagation.
TEZ-1749. Increase test timeout for TestLocalMode.testMultipleClientsWithSession
TEZ-1750. Add a DAGScheduler which schedules tasks only when sources have been scheduled.
TEZ-1761. TestRecoveryParser::testGetLastInProgressDAG fails in similar manner to TEZ-1686.
TEZ-1770. Handle ConnectExceptions correctly when establishing connections to an NM which may be down.
TEZ-1774. AppLaunched event for Timeline does not have start time set.
TEZ-1780. tez-api is missing jersey dependencies.
TEZ-1796. Use of DeprecationDelta broke build against 2.2 Hadoop.
TEZ-1818. Problem loading tez-api-version-info.properties in case current context classloader
in not pointing to Tez jars.
TEZ-1808. Job can fail since name of intermediate files can be too long in specific situation.
Release 0.5.2: 2014-11-07
INCOMPATIBLE CHANGES
TEZ-1666. UserPayload should be null if the payload is not specified.
0.5.1 client cannot talk to 0.5.2 AMs (TEZ-1666 and TEZ-1664).
context.getUserPayload can now return null, apps may need to add defensive code.
TEZ-1699. Vertex.setParallelism should throw an exception for invalid
invocations
TEZ-1700. Replace containerId from TaskLocationHint with [TaskIndex+Vertex]
based affinity
ALL CHANGES:
TEZ-1620. Wait for application finish before stopping MiniTezCluster
TEZ-1621. Should report error to AM before shuting down TezChild
TEZ-1634. Fix compressed IFile shuffle errors
TEZ-1648. Update website after 0.5.1
TEZ-1614. Use setFromConfiguration() in SortMergeJoinExample to demonstrate the usage
TEZ-1641. Add debug logs in VertexManager to help debugging custom VertexManagerPlugins
TEZ-1645. Add support for specifying additional local resources via config.
TEZ-1646. Add support for augmenting classpath via configs.
TEZ-1647. Issue with caching of events in VertexManager::onRootVertexInitialized.
TEZ-1470. Recovery fails due to TaskAttemptFinishedEvent being recorded multiple times for the same task.
TEZ-1649. ShuffleVertexManager auto reduce parallelism can cause jobs to hang indefinitely.
TEZ-1566. Reduce log verbosity.
TEZ-1083. Enable IFile RLE for DefaultSorter.
TEZ-1637. Improved shuffle error handling across NM restarts.
TEZ-1479. Disambiguate (refactor) between ShuffleInputEventHandlers and Fetchers.
TEZ-1665. DAGScheduler should provide a priority range instead of an exact
priority
TEZ-1632. NPE at TestPreemption.testPreemptionWithoutSession
TEZ-1674. Rename configuration parameters related to counters / memory scaling.
TEZ-1176. Set parallelism should end up sending an update to ATS if numTasks are updated at run-time.
TEZ-1658. Additional data generation to Timeline for UI.
TEZ-1676. Fix failing test in TestHistoryEventTimelineConversion.
TEZ-1673. Update the default value for allowed node failures, num events per heartbeat
and counter update interval.
TEZ-1462. Remove unnecessary SuppressWarnings.
TEZ-1633. Fixed expected values in TestTaskRecovery.testRecovery_OneTAStarted.
TEZ-1669. yarn-swimlanes.sh throws error post TEZ-1556.
TEZ-1682. Tez AM hangs at times when there are task failures.
TEZ-1683. Do ugi::getGroups only when necessary when checking ACLs.
TEZ-1584. Restore counters from DAGFinishedEvent when DAG is completed.
TEZ-1525. BroadcastLoadGen testcase.
TEZ-1686. TestRecoveryParser.testGetLastCompletedDAG fails sometimes
TEZ-1667. Add a system test for InitializerEvents.
TEZ-1668. InputInitializers should be able to register for Vertex state updates in the constructor.
TEZ-1656. Grouping of splits should maintain the original ordering of splits
within a group
TEZ-1396. Grouping should generate consistent groups when given the same set
of splits
TEZ-1210. TezClientUtils.localizeDagPlanAsText() needs to be fixed for
session mode
TEZ-1629. Replace ThreadPool's default RejectedExecutionHandler in ContainerLauncherImpl to void
abort when AM shutdown.
TEZ-1643. DAGAppMaster kills DAG & shuts down, when RM is restarted.
TEZ-1684. upgrade mockito to latest release.
TEZ-1567. Avoid blacklisting nodes when the disable blacklisting threshold is about to be hit.
TEZ-1267. Exception handling for VertexManager.
TEZ-1688. Add applicationId as a primary filter for all Timeline data for easier export.
TEZ-1141. DAGStatus.Progress should include number of failed and killed attempts.
TEZ-1424. Fixes to DAG text representation in debug mode.
TEZ-1590. Fetchers should not report failures after the Processor on the task completes.
TEZ-1542. Fix a Local Mode crash on concurrentModificationException.
TEZ-1638. Fix Missing type parametrization in runtime Input/Output configs.
TEZ-1596. Secure Shuffle utils is extremely expensive for fast queries.
TEZ-1712. SSL context gets destroyed too soon after downloading data from one of the vertices.
TEZ-1710. Add support for cluster default AM/task launch opts.
TEZ-1713. tez.lib.uris should not require the paths specified to be fully qualified.
TEZ-1715. Fix use of import java.util.* in MultiMRInput.
TEZ-1664. Add checks to ensure that the client and AM are compatible.
TEZ-1689. Exception handling for EdgeManagerPlugin.
TEZ-1701. ATS fixes to flush all history events and also using batching.
TEZ-792. Default staging path should have user name.
TEZ-1689. addendum - fix unit test failure.
TEZ-1666. UserPayload should be null if the payload is not specified.
0.5.1 client cannot talk to 0.5.2 AMs (TEZ-1666 and TEZ-1664).
context.getUserPayload can now return null, apps may need to add defensive code.
TEZ-1699. Vertex.setParallelism should throw an exception for invalid
invocations
TEZ-1700. Replace containerId from TaskLocationHint with [TaskIndex+Vertex]
based affinity
TEZ-1716. Additional ATS data for UI.
TEZ-1722. DAG should be related to Application Id in ATS data.
TEZ-1711. Don't cache outputSpecList in VertexImpl.getOutputSpecList(taskIndex)
TEZ-1703. Exception handling for InputInitializer.
TEZ-1698. Cut down on ResourceCalculatorProcessTree overheads in Tez.
TEZ-1703. addendum - fix flaky test.
TEZ-1725. Fix nanosecond to millis conversion in TezMxBeanResourceCalculator.
TEZ-1726. Build broken against Hadoop-2.6.0 due to change in NodeReport.
TEZ-1579. MR examples should be setting mapreduce.framework.name to yarn-tez.
TEZ-1731. OnDiskMerger can end up clobbering files across tasks with LocalDiskFetch enabled.
TEZ-1735. Allow setting basic info per DAG for Tez UI.
TEZ-1728. Remove local host name from Fetcher thread name.
TEZ-1547. Make use of state change notifier in VertexManagerPlugins and fix
TEZ-1494 without latency penalty
Release 0.5.1: 2014-10-02
INCOMPATIBLE CHANGES
TEZ-1488. Rename HashComparator to ProxyComparator and implement in TezBytesComparator
TEZ-1578. Remove TeraSort from Tez codebase.
TEZ-1499. Add SortMergeJoinExample to tez-examples
TEZ-1539. Change InputInitializerEvent semantics to SEND_ONCE_ON_TASK_SUCCESS
TEZ-1571. Add create method for DataSinkDescriptor.
ALL CHANGES
TEZ-1544. Link to release artifacts for 0.5.0 does not point to a specific link for 0.5.0.
TEZ-1559. Add system tests for AM recovery.
TEZ-850. Recovery unit tests.
TEZ-853. Support counters recovery.
TEZ-1345. Add checks to guarantee all init events are written to recovery to consider vertex initialized.
TEZ-1575. MRRSleepJob does not pick MR settings for container size and java opts.
TEZ-1488. Rename HashComparator to ProxyComparator and implement in TezBytesComparator
TEZ-1578. Remove TeraSort from Tez codebase.
TEZ-1569. Add tests for preemption
TEZ-1580. Change TestOrderedWordCount to optionally use MR configs.
TEZ-1524. Resolve user group information only if ACLs are enabled.
TEZ-1581. GroupByOrderByMRRTest no longer functional.
TEZ-1157. Optimize broadcast shuffle to download data only once per host.
TEZ-1594. Initial TezUI into TEZ-8 branch
TEZ-1595. Document timeline server setup for use with the Tez UI
TEZ-1157. Optimize broadcast shuffle to download data only once per host.
TEZ-1607. support mr envs in mrrsleep and testorderedwordcount
TEZ-1499. Add SortMergeJoinExample to tez-examples
TEZ-1613. Decrease running time for TestAMRecovery
TEZ-1240. Add system test for propagation of diagnostics for errors
TEZ-1618. LocalTaskSchedulerService.getTotalResources() and getAvailableResources() can get
negative if JVM memory is larger than 2GB
TEZ-1611. Change DataSource/Sink to be able to supply URIs for credentials
TEZ-1592. Vertex should wait for all initializers to finish before moving to INITED state
TEZ-1612. ShuffleVertexManager's EdgeManager should not hard code source num tasks
TEZ-1555. TestTezClientUtils.validateSetTezJarLocalResourcesDefinedButEmpty
failing on Windows
TEZ-1609. Add hostname to logIdentifiers of fetchers for easy debugging
TEZ-1494. DAG hangs waiting for ShuffleManager.getNextInput()
TEZ-1515. Remove usage of ResourceBundles in Counters.
TEZ-1527. Fix indentation of Vertex status in DAGClient output.
TEZ-1536. Fix spelling typo "configurartion" in TezClientUtils.
TEZ-1310. Update website documentation framework
TEZ-1447. Provide a mechanism for InputInitializers to know about Vertex state changes.
TEZ-1362. Remove DAG_COMPLETED in DAGEventType.
TEZ-1519. TezTaskRunner should not initialize TezConfiguration in TezChild.
TEZ-1534. Make client side configs available to AM and tasks.
TEZ-1574. Support additional formats for the tez deployed archive
TEZ-1563. TezClient.submitDAGSession alters DAG local resources regardless
of DAG submission
TEZ-1585. Memory leak in tez session mode.
TEZ-1533. Request Events more often if a complete set of events is received by a task.
TEZ-1587. Some tez-examples fail in local mode.
TEZ-1597. ImmediateStartVertexManager should handle corner case of vertex having zero tasks.
TEZ-1495. ATS integration for TezClient
TEZ-1553. Multiple failures in testing path-related tests in
TestTezCommonUtils for Windows
TEZ-1598. DAGClientTimelineImpl uses ReflectiveOperationException (which has JDK 1.7 dependency)
TEZ-1599. TezClient.preWarm() is not enabled
TEZ-1550. TestEnvironmentUpdateUtils.testMultipleUpdateEnvironment fails on
Windows
TEZ-1554. Failing tests in TestMRHelpers related to environment on Windows
TEZ-978. Enhance auto parallelism tuning for queries having empty outputs or data skewness
TEZ-1433. Invalid credentials can be used when a DAG is submitted to a
session which has timed out
TEZ-1624. Flaky tests in TestContainerReuse due to race condition in DelayedContainerManager thread
Release 0.5.0: 2014-09-03
INCOMPATIBLE CHANGES
TEZ-1038. Move TaskLocationHint outside of VertexLocationHint.
TEZ-960. VertexManagerPluginContext::getTotalAVailableResource() changed to
VertexManagerPluginContext::getTotalAvailableResource()
TEZ-1025. Rename tez.am.max.task.attempts to tez.am.task.max.failed.attempts
TEZ-1018. VertexManagerPluginContext should enable assigning locality to
scheduled tasks
TEZ-1169. Allow numPhysicalInputs to be specified for RootInputs.
TEZ-1131. Simplify EdgeManager APIs
TEZ-1127. Add TEZ_TASK_JAVA_OPTS and TEZ_ENV configs to specify values from
config
TEZ-692. Unify job submission in either TezClient or TezSession
TEZ-1130. Replace confusing names on Vertex API
TEZ-1213. Fix parameter naming in TezJobConfig.
- Details at https://issues.apache.org/jira/browse/TEZ-1213?focusedCommentId
=14039381&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpa
nel#comment-14039381
TEZ-1080, TEZ-1272, TEZ-1279, TEZ-1266. Change YARNRunner to use EdgeConfigs.
- Removes separation of runtime configs into input/ouput configs. Also
refactors public methods used for this conversion.
TEZ-696. Remove implicit copying of processor payload to input and output
TEZ-1269. TaskScheduler prematurely releases containers
TEZ-857. Split Input/Output interfaces into user/framework components.
TEZ-866. Add a TezMergedInputContext for MergedInputs
TEZ-1137. Move TezJobConfig to runtime-library and rename to
TezRuntimeConfiguration
TEZ-1134. InputInitializer and OutputCommitter implicitly use payloads of
the input and output
TEZ-1312. rename vertex.addInput/Output() to vertex.addDataSource/Sink()
TEZ-1311. get sharedobjectregistry from the context instead of a static
TEZ-1300. Change deploy mechanism for Tez to be based on a tarball which
includes Hadoop libs.
TEZ-1278. TezClient#waitTillReady() should not swallow interrupts
TEZ-1058. Replace user land interfaces with abstract classes
TEZ-1303. Change Inputs, Outputs, InputInitializer, OutputCommitter, VertexManagerPlugin, EdgeManager
to require constructors for creation, and remove the initialize methods.
TEZ-1133. Remove some unused methods from MRHelpers.
TEZ-1346. Change Processor to require context constructors for creation, and remove the requirement of the initialize method requiring the context.
TEZ-1041. Use VertexLocationHint consistently everywhere in the API
TEZ-1057. Replace interfaces with abstract classes for
Processor/Input/Output classes
TEZ-1351. MROutput needs a flush method to ensure data is materialized for
FileOutputCommitter
TEZ-1317. Simplify MRinput/MROutput configuration
TEZ-1379. Allow EdgeConfigurers to accept Configuration for Comparators. Change the way partitioner, comparator, combiner confs are set (from Hadoop Configuration to Map). Rename specific Input/Output classes from *Configuration to *Configurer.
TEZ-1382. Change ObjectRegistry API to allow for future extensions
TEZ-1386. TezGroupedSplitsInputFormat should not need to be setup to enable grouping.
TEZ-1394. Create example code for OrderedWordCount
TEZ-1372. Fix preWarm to work after recent API changes
TEZ-1237. Consolidate naming of API classes
TEZ-1407. Move MRInput related methods out of MRHelpers and consolidate.
TEZ-1194. Make TezUserPayload user facing for payload specification
TEZ-1347. Consolidate MRHelpers.
TEZ-1072. Consolidate monitoring APIs in DAGClient
TEZ-1410. DAGClient#waitForCompletion() methods should not swallow interrupts
TEZ-1416. tez-api project javadoc/annotations review and clean up
TEZ-1425. Move constants to TezConstants
TEZ-1388. mvn site is slow and generates errors
TEZ-1432. Rename property to cancel delegation tokens on app completion (tez.am.am.complete.cancel.delegation.tokens)
TEZ-1320. Remove getApplicationId from DAGClient
TEZ-1427. Change remaining classes that are using byte[] to UserPayload
TEZ-1418. Provide Default value for TEZ_AM_LAUNCH_ENV and TEZ_TASK_LAUNCH
TEZ-1438. Annotate add java doc for tez-runtime-library and tez-mapreduce
TEZ-1055. Rename tez-mapreduce-examples to tez-examples
TEZ-1132. Consistent naming of Input and Outputs
TEZ-1423. Ability to pass custom properties to keySerializer for
OnFileUnorderedPartitionedKVOutput
TEZ-1426. Create configuration helpers for ShuffleVertexManager and
TezGrouping code
TEZ-1390. Replace byte[] with ByteBuffer as the type of user payload in the API
TEZ-1417. Rename Configurer* to Config/ConfigBuilder
TEZ-1450. Documentation of TezConfiguration
TEZ-1231. Clean up TezRuntimeConfiguration
TEZ-1246. Replace constructors with create() methods for DAG, Vertex, Edge etc in the API
TEZ-1455. Replace deprecated junit.framework.Assert with org.junit.Assert
TEZ-1465. Update and document IntersectExample. Change name to JoinExample
TEZ-1449. Change user payloads to work with a byte buffer
TEZ-1469. AM/Session LRs are not shipped to vertices in new API use-case
TEZ-1472. Separate method calls for creating InputDataInformationEvent with
serialized/unserialized payloads
TEZ-1485. Disable node blacklisting and ATS in AM for local mode
TEZ-1463. Remove dependency on private class org.apache.hadoop.util.StringUtils
TEZ-1476. DAGClient waitForCompletion output is confusing
TEZ-1490. dagid reported is incorrect in TezClient.java
TEZ-1500. DAG should be created via a create method
TEZ-1509. Set a useful default value for java opts
ALL CHANGES
TEZ-1516. Log transfer rates for broadcast fetch. (sseth)
TEZ-1511. MROutputConfigBuilder sets OutputFormat as String class if OutputFormat is not provided (bikas)
TEZ-1509. Set a useful default value for java opts (bikas)
TEZ-1517. Avoid sending routed events via the AsyncDispatcher. (sseth)
TEZ-1510. Add missed file. TezConfiguration should not add tez-site.xml as a default resource. (hitesh)
TEZ-1510. Addendum patch. TezConfiguration should not add tez-site.xml as a default resource. (hitesh)
TEZ-1510. TezConfiguration should not add tez-site.xml as a default resource. (hitesh)
TEZ-1501. Add a test dag to generate load on the getTask RPC. (sseth)
TEZ-1481. Flaky test : org.apache.tez.dag.api.client.TestDAGClientHandler.testDAGClientHandler (Contributed by Alexander Pivovarov)
TEZ-1512. VertexImpl.getTask(int) can be CPU intensive when lots of tasks are present in the vertex
TEZ-1492. IFile RLE not kicking in due to bug in BufferUtils.compare()
TEZ-1496. Multi MR inputs can not be configured without accessing internal proto structures (Siddharth Seth via bikas)
TEZ-1493. Tez examples sometimes fail in cases where AM recovery kicks in. (Jeff Zhang via hitesh)
TEZ-1038. Move TaskLocationHint outside of VertexLocationHint. (Alexander Pivovarov via hitesh)
TEZ-1475. Fix HDFS commands in INSTALL.txt (bikas)
TEZ-1500. DAG should be created via a create method (Siddharth Seth via bikas)
TEZ-1430. Javadoc generation should not generate docs for classes annotated as private. (Jonathan Eagles via hitesh)
TEZ-1498. Usage info is not printed when wrong number of arguments is provided for JoinExample. (Jeff Zhang via hitesh)
TEZ-1486. Event routing should not throw an exception if the EdgePlugin does not generate a routing table in cases where the destination vertex has a parallelism of 0. (sseth)
TEZ-1490. dagid reported is incorrect in TezClient.java (jeagles)
TEZ-1476. DAGClient waitForCompletion output is confusing (jeagles)
TEZ-1471. Additional supplement for TEZ local mode document. Contributed by Chen He.
TEZ-1360. Provide vertex parallelism to each vertex task. Contributed by Gopal V, Johannes Zillmann and Rajesh Balamohan.
TEZ-1463. Remove dependency on private class org.apache.hadoop.util.StringUtils (Alexander Pivovarov via jeagles)
TEZ-1448. Make WeightedScalingMemoryDistributor as the default memory distributor (Rajesh Balamohan)
TEZ-1485. Disable node blacklisting and ATS in AM for local mode (jeagles)
TEZ-1446. Move the fetch code for local disk fetch from data movement event handlers to fetcher. Contributed by Prakash Ramachandran.
TEZ-1487. Switch master to 0.6.0-SNAPSHOT. (hitesh)
TEZ-1474. detect missing native libraries for compression at the beginning of a task rather than at the end. Contributed by Prakash Ramachandran.
TEZ-1436. Fix javadoc warnings (Jonathan Eagles via bikas)
TEZ-1472. Separate method calls for creating InputDataInformationEvent with serialized/unserialized payloads (Siddharth Seth via bikas)
TEZ-1469. AM/Session LRs are not shipped to vertices in new API use-case (bikas)
TEZ-1464 Addendum. Update INSTALL.txt (bikas)
TEZ-1464. Update INSTALL.txt (bikas)
TEZ-1449. Change user payloads to work with a byte buffer (Siddharth Seth via bikas)
TEZ-1325. RecoveryParser can find incorrect last DAG ID. (Jeff Zhang via hitesh)
TEZ-1466. Fix JDK8 builds of Tez (gopalv)
TEZ-1251. Fix website to not display latest snapshot version in header. (Alexander Pivovarov via hitesh)
TEZ-1465. Update and document IntersectExample. Change name to JoinExample (bikas)
TEZ-1458. org.apache.tez.common.security.Groups does not compile against hadoop-2.2.0 anymore. (hitesh)
TEZ-1455. Replace deprecated junit.framework.Assert with org.junit.Assert (Alexander Pivovarov via jeagles)
TEZ-1454. Remove unused imports (Alexander Pivovarov via bikas)
TEZ-1415. Merge various Util classes in Tez (Alexander Pivovarov via bikas)
TEZ-1461. Add public key to KEYS (bikas)
TEZ-1452. Add license and notice to jars (bikas)
TEZ-1456. Fix typo in TestIFile.testWithRLEMarker (Contributed by Alexander Pivovarov)
TEZ-1246. Replace constructors with create() methods for DAG, Vertex, Edge etc in the API. (sseth)
TEZ-1453. Fix rat check for 0.5 (bikas)
TEZ-1231. Clean up TezRuntimeConfiguration (bikas)
TEZ-1450. Documentation of TezConfiguration (bikas)
TEZ-1417. Rename *Configurer to ConfigBuilder/Config. (sseth)
TEZ-1349. Add documentation for LocalMode usage. (sseth)
TEZ-1390. Replace byte[] with ByteBuffer as the type of user payload in the API. Contributed by Tsuyoshi OZAWA.
TEZ-1395. Fix failure in IFile handling of compressed data. (Rajesh Balamohan via hitesh)
TEZ-1445. Add more logging to catch shutdown handler race conditions. (hitesh)
TEZ-1426. Create configuration helpers for ShuffleVertexManager and TezGrouping code (Rajesh Balamohan via bikas)
TEZ-1439. IntersectDataGen/Example/Validate should move back to tez-examples. (hitesh)
TEZ-1423. Ability to pass custom properties to keySerializer for OnFileUnorderedPartitionedKVOutput (Siddharth Seth via bikas)
TEZ-1132. Consistent naming of Input and Outputs (bikas)
TEZ-1400. Reducers stuck when enabling auto-reduce parallelism
TEZ-1055 addendum. Rename tez-mapreduce-examples to tez-examples (Hitesh Shah via bikas)
TEZ-1055. Rename tez-mapreduce-examples to tez-examples (Hitesh Shah via bikas)
TEZ-1438. Annotate add java doc for tez-runtime-library and tez-mapreduce. (bikas via hitesh)
TEZ-1411. Address initial feedback on swimlanes (gopalv)
TEZ-1418. Provide Default value for TEZ_AM_LAUNCH_ENV and TEZ_TASK_LAUNCH (Subroto Sanyal via bikas)
TEZ-1065 addendum-1 to fix broken test (bikas)
TEZ-1435. Fix unused imports. (hitesh)
TEZ-1434. Make only wait apis in TezClient to throw InterruptedException. (hitesh)
TEZ-1427. Change remaining classes that are using byte[] to UserPayload. (sseth)
TEZ-1429. Avoid sysexit in the DAGAM in case of local mode. (sseth)
TEZ-1338. Support submission of multiple applications with LocalRunner from within the same JVM. (sseth)
TEZ-1334. Annotate all non public classes in tez-runtime-library with @private. (hitesh)
TEZ-1320. Remove getApplicationId from DAGClient (Jonathan Eagles via bikas)
TEZ-1065 addendum to fix broken test (bikas)
TEZ-1431. Fix use of synchronized for certain functions in TezClient. (hitesh)
TEZ-1065. DAGStatus.getVertexStatus and other vertex related API's should maintain vertex order (Jeff Zhang via bikas)
TEZ-1432. TEZ_AM_CANCEL_DELEGATION_TOKEN is named inorrectly. (sseth)
TEZ-1388. mvn site is slow and generates errors (jeagles)
TEZ-1409. Change MRInputConfigurer, MROutputConfigurer to accept complete configurations. (sseth)
TEZ-1425. Move constants to TezConstants (bikas)
TEZ-1416. tez-api project javadoc/annotations review and clean up (bikas)
TEZ-671. Support View/Modify ACLs for DAGs. (hitesh)
TEZ-1413. Fix build for TestTezClientUtils.testLocalResourceVisibility (Prakash Ramachandran via bikas)
TEZ-1422. Use NetUtils to create the bind address for the client, which allows clients to setup static address resolution. Contributed by Johannes Zillmann.
TEZ-1410. DAGClient#waitForCompletion() methods should not swallow interrupts. Contributed by Johannes Zillmann.
TEZ-1419. Release link broken on website for 0.4.1 release. (hitesh)
TEZ-1072. Consolidate monitoring APIs in DAGClient (jeagles)
TEZ-1420. Remove unused classes - LocalClientProtocolProviderTez, LocalJobRunnerMetricsTez, LocalJobRunnerTez. (sseth)
TEZ-1330. Create a default dist target which contains jars. (sseth)
TEZ-1414. Disable TestTezClientUtils.testLocalResourceVisibility to make builds pass(bikas)
TEZ-1347. Consolidate MRHelpers. (sseth)
TEZ-1402. MRoutput configurer should allow disabling the committer (bikas)
TEZ-817. TEZ_LIB_URI are always uploaded as public Local Resource (Prakash Ramachandran via bikas)
TEZ-1404. groupCommitInProgress in RecoveryTransition of DAGImpl is not set correctly. (Jeff Zhang via hitesh)
TEZ-1024. Fix determination of failed attempts in recovery. (Jeff Zhang via hitesh)
TEZ-1403. oah.mapred.Partitioner is not configured by JobConf. Contributed by Navis.
TEZ-1399. Add an example to show session usage (bikas)
TEZ-1194. Make TezUserPayload user facing for payload specification (Tsuyoshi Ozawa and bikas)
TEZ-1393. user.dir should not be reset in LocalMode. (sseth)
TEZ-1216. Clean up the staging directory when the application completes. (hitesh) This closes #3
TEZ-1407. Move MRInput related methods out of MRHelpers and consolidate. (sseth)
TEZ-1205. Remove profiling keyword from APIs/configs
TEZ-1405. TestSecureShuffle is slow (Rajesh Balamohan via bikas)
TEZ-1237. Consolidate naming of API classes (bikas)
TEZ-1391. Setup IGNORE_LIB_URIS correctly for Local Mode. (sseth)
TEZ-1318 addendum. Simplify Vertex Constructor (bikas)
TEZ-1318. Simplify Vertex Constructor (bikas)
TEZ-1372. Remove author tag from previous commit (bikas)
TEZ-1372. Fix preWarm to work after recent API changes (bikas)
TEZ-1372. Fix preWarm to work after recent API changes (bikas)
TEZ-1394. Create example code for OrderedWordCount (bikas)
TEZ-1394. Create example code for OrderedWordCount (bikas)
TEZ-1392. Fix MRRSleepJob failure. (sseth)
TEZ-1386. TezGroupedSplitsInputFormat should not need to be setup to enable grouping. (sseth)
TEZ-1385. Disk Direct fails for MapOutput when trying to use OnDiskMerger. Contributed by Prakash Ramachandran.
TEZ-1382. Change ObjectRegistry API to allow for future extensions (bikas)
TEZ-1332. Swimlane diagrams from tez AM logs (gopalv)
TEZ-1368. TestSecureShuffle failing
TEZ-1379. Allow EdgeConfigurers to accept Configuration for Comparators. Change the way partitioner, comparator, combiner confs are set (from Hadoop Configuration to Map). Rename specific Input/Output classes from *Configuration to *Configurer. (sseth)
TEZ-1317. Simplify MRinput/MROutput configuration (bikas)
TEZ-1351. MROutput needs a flush method to ensure data is materialized for FileOutputCommitter (bikas)
TEZ-1368 is fixed. (hitesh)
TEZ-1057. Replace interfaces with abstract classes for Processor/Input/Output classes (bikas)
TEZ-1041. Use VertexLocationHint consistently everywhere in the API (bikas)
TEZ-1343. Bypass the Fetcher and read directly from the local filesystem if source task ran on the same host. Contributed by Prakash Ramachandran.
TEZ-1365. Local mode should ignore tez.lib.uris, and set a config for the AM to be aware of session mode. (sseth)
TEZ-1355. Read host and shuffle meta-information for events only if data is generated by Outputs. Contributed by Jonathan Eagles.
TEZ-1342. Fix a bug which caused tez.am.client.am.port-range to not take effect. Contributed by Jeff Zhang.
TEZ-870. Change LocalContainerLauncher to handle multiple threads, improve error reporting and inform other components about container completion. (sseth)
TEZ-1352. HADOOP_CONF_DIR should be in the classpath for containers. (sseth)
TEZ-1238. Display more clear diagnostics info on client side on task failures. (Jeff Zhang via hitesh)
TEZ-1354. Fix NPE in FilterByWordOutputProcessor. Contributed by Jonathan Eagles.
TEZ-1322. OrderedWordCount broken in master branch. (hitesh)
TEZ-1341. IFile append() has string concats leading to memory pressure
TEZ-1346. Change Processor to require context constructors for creation, and remove the requirement of the initialize method requiring the context. (sseth)
TEZ-1064. Restore dagName Set for duplicate detection in recovered AMs. (Jeff Zhang via hitesh)
TEZ-1133 as incompatible.
TEZ-1133. Remove some unused methods from MRHelpers. Contributed by Chen He.
TEZ-1303. Change Inputs, Outputs, InputInitializer, OutputCommitter, VertexManagerPlugin, EdgeManager to require constructors for creation, and remove the initialize methods. (sseth)
TEZ-1276. Remove unnecessary TaskAttemptEventType TA_FAIL_REQUEST. (Jeff Zhang via hitesh)
TEZ-1326. AMStartedEvent should not be recovery event. (Jeff Zhang via hitesh)
TEZ-1333. Flaky test: TestOnFileSortedOutput fails in jenkins server with OOM
TEZ-717. Client changes for local mode DAG submission. Contributed by Jonathan Eagles.
TEZ-707. Add a LocalContainerLauncher. Contributed by Chen He.
TEZ-1058. Replace user land interfaces with abstract classes (bikas)
TEZ-1324. OnFileSortedOutput: send host/port/pathComponent details only when one of the partitions has data
TEZ-1328. Move EnvironmentUpdateUtils to tez-common
TEZ-1278. TezClient#waitTillReady() should not swallow interrupts. Contributed by Johannes Zillmann.
TEZ-1305: Log job tracking url (rohini)
TEZ-1257. Error on empty partition when using OnFileUnorderedKVOutput and ShuffledMergedInput
TEZ-1300. Change default tez classpath to not include hadoop jars from the cluster. (sseth)
TEZ-1321. Remove methods annotated as @Private from TezClient and DAGClient. (sseth)
TEZ-1288. Create FastTezSerialization as an optional feature (rajesh)
TEZ-1304. Abstract out client interactions with YARN. Contributed by Jonathan Eagles.
TEZ-1306. Remove unused ValuesIterator. Contributed by Jonathan Eagles.
TEZ-1127 addendum for changing tez.am.java.opts. Add TEZ_TASK_JAVA_OPTS and TEZ_ENV configs to specify values from config (bikas)
TEZ-1311. get sharedobjectregistry from the context instead of a static (bikas)
TEZ-1312. rename vertex.addInput/Output() to vertex.addDataSource/Sink() (Chen He via bikas)
TEZ-1134. InputInitializer and OutputCommitter implicitly use payloads of the input and output (bikas)
TEZ-1309. Use hflush instead of hsync in recovery log. (hitesh)
TEZ-1137. Move TezJobConfig to runtime-library and rename to TezRuntimeConfiguration (bikas)
TEZ-1247. Allow DAG.verify() to be called multiple times (Jeff Zhang via bikas)
TEZ-1299. Get rid of unnecessary setter override in EntityDescriptors. (sseth)
TEZ-866. Add a TezMergedInputContext for MergedInputs (bikas)
TEZ-1296. commons-math3 dependency (bikas)
TEZ-1301. Fix title of pages in docs. (hitesh)
TEZ-811. Addendum. Update page title. (hitesh)
TEZ-811. Update docs on how to contribute to Tez. (hitesh)
TEZ-1242. Icon. Logos for Tez (Harshad P Dhavale via bikas)
TEZ-1298. Add parameterized constructor capabilities in ReflectionUtils. Contributed by Jonathan Eagles.
TEZ-1242. Logos for Tez (Harshad P Dhavale via bikas)
TEZ-1242. Logos for Tez (Harshad P Dhavale via bikas)
TEZ-1295. Modify the tez-dist-full build target to include hadoop libraries. Also makes the tez direct dependencies explicit in the poms. (sseth)
TEZ-857. Split Input/Output interfaces into user/framework components. (sseth)
TEZ-1290. Make graduation related changes. (hitesh)
TEZ-1269. TaskScheduler prematurely releases containers (bikas)
TEZ-1269. TaskScheduler prematurely releases containers (bikas)
TEZ-1119. Support display of user payloads in Tez UI. (hitesh)
TEZ-1089. Change CompositeDataMovementEvent endIndex to a count of number of events. Contributed by Chen He.
TEZ-696. Remove implicit copying of processor payload to input and output (bikas)
TEZ-1285. Add Utility for Modifying Environment Variables. Contributed by Jonathan Eagles and Oleg Zhurakousky.
TEZ-1287. TestJavaProfilerOptions is missing apache license header. Contributed by Jonathan Eagles.
TEZ-1260. Allow KeyValueWriter to support writing list of values
TEZ-1244. Fix typo in RootInputDataInformationEvent javadoc. Contributed by Chen He.
TEZ-1266. Create *EdgeConfigurer.createDefaultCustomEdge() and force setting partitioners where applicable. (sseth)
TEZ-1279. Rename *EdgeConfiguration to *EdgeConfigurer. (sseth)
TEZ-1272. Change YARNRunner to make use of EdgeConfigurations. (sseth)
TEZ-1130. Replace confusing names on Vertex API (bikas)
TEZ-1228. Define a memory & merge optimized vertex-intermediate file format for Tez
TEZ-1076. Allow events to be sent to InputInitializers. (sseth)
TEZ-657. Tez should process the Container exit status - specifically when the RM preempts a container (bikas)
TEZ-1262. Change Tez examples to use Edge configs. (sseth)
TEZ-1241 Consistent getter for staging dir (kamrul)
TEZ-1131 addendum for missing fix. Simplify EdgeManager APIs (bikas)
TEZ-1118. Tez with container reuse reports negative CPU usage. Contributed by Robert Grandl.
TEZ-1131 (bikas)
TEZ-1080. Add specific Configuration APIs for non MR based Inputs / Outputs. (sseth)
TEZ-225. Tests for DAGClient (Jeff Zhang via bikas)
TEZ-1258. Remove unused class JobStateInternal. Contributed by Jeff Zhang.
TEZ-1253. Remove unused VertexEventTypes. Contributed by Jeff Zhang.
TEZ-1170 addendum to remove unnecessary transitions. Simplify Vertex Initializing transition (bikas)
TEZ-692. Unify job submission in either TezClient or TezSession (bikas)
TEZ-1163. Tez Auto Reducer-parallelism throws Divide-by-Zero
TEZ-699. Have sensible defaults for java opts. (hitesh)
TEZ-387. Move DAGClientHandler into its own class (Jeff Zhang via bikas)
TEZ-1234. Replace Interfaces with Abstract classes for VertexManagerPlugin and EdgeManager. (hitesh)
TEZ-1218. Make TaskScheduler an Abstract class instead of an Inteface. Contributed by Jeff Zhang.
TEZ-1127. Add TEZ_TASK_JAVA_OPTS and TEZ_ENV configs to specify values from config
TEZ-1214. Diagnostics of Vertex is missing when constructing TimelineEntity. (Jeff Zhang via hitesh)
TEZ-106. TaskImpl does not hold any diagnostic information that can be emitted to history. (Jeff Zhang via hitesh)
TEZ-1219. Addendum. Fix roles. (hitesh)
TEZ-1219. Update team list to match incubator status page. (hitesh)
TEZ-1213. Fix parameter naming in TezJobConfig. (sseth)
TEZ-1168. Add MultiMRInput, which can process multiple splits, and returns individual readers for each of these. (sseth)
TEZ-1106. Tez framework should use a unique subdir for staging data. (Mohammad Kamrul Islam via hitesh)
TEZ-1042.Stop re-routing stdout, stderr for tasks and AM. (sseth)
TEZ-1208. Log time taken to connect/getInputStream to a http source in fetcher. Contributed by Rajesh Balamohan.
TEZ-1172. Allow multiple Root Inputs to be specified per Vertex. (sseth)
TEZ-1170 Simplify Vertex Initializing transition (bikas)
TEZ-1193. Allow 'tez.lib.uris' to be overridden (Oleg Zhurakousky via bikas)
TEZ-1032. Allow specifying tasks/vertices to be profiled. (Rajesh Balamohan via hitesh)
TEZ-1131. Simplify EdgeManager APIs
TEZ-1169 addendum to modify the incompatible change list in CHANGES.txt
TEZ-1169. Allow numPhysicalInputs to be specified for RootInputs. (sseth)
TEZ-1192. Fix loop termination in TezChild. Contributed by Oleg Zhurakousky.
TEZ-1178. Prevent duplicate ObjectRegistryImpl inits in TezChild. (gopalv)
TEZ-1199. EdgeVertexName in EventMetaData can be null. (hitesh)
TEZ-1196. FaultToleranceTestRunner should allow passing generic options from cli (Karam Singh via tassapola)
TEZ-1162. The simple history text files now use ^A\n as their line endings.
TEZ-1164. Only events for tasks should be buffered in Initializing state (bikas)
TEZ-1171. Vertex remains in INITED state if all source vertices start while the vertex was in INITIALIZING state (bikas)
TEZ-373. Create UserPayload class for internal code (Tsuyoshi OZAWA via bikas)
TEZ-1151. Vertex should stay in initializing state until custom vertex manager sets the parallelism (bikas)
TEZ-1145. Vertices should not start if they have uninitialized custom edges (bikas)
TEZ-1143 (addendum). 1-1 source split event should be handled in Vertex.RUNNING and Vertex.INITED state (bikas)
TEZ-1116. Refactor YarnTezDAGChild to be testable and usable for LocalMode. (sseth)
TEZ-1143. 1-1 source split event should be handled in Vertex.RUNNING state (bikas)
TEZ-800. One-one edge with parallelism -1 fails if source vertex parallelism is not -1 as well (bikas)
TEZ-1154. tez-mapreduce-examples should depend on yarn-api. (hitesh)
TEZ-1090. Micro optimization - Remove duplicate updateProcessTree() in TaskCounterUpdater. (Rajesh Balamohan via hitesh)
TEZ-1027. orderedwordcount needs to respect tez.staging-dir property. (Rekha Joshi via hitesh)
TEZ-1150. Replace String EdgeId with Edge in the Vertex (bikas)
TEZ-1066. Generate events to integrate with YARN timeline server. (hitesh)
TEZ-1140. TestSecureShuffle leaves behind test data dirs. (hitesh)
TEZ-1039. Add Container locality to TaskScheduler (bikas)
TEZ-1139. Add a test for IntersectDataGen and IntersectValidate. (sseth)
TEZ-1126. Add a data generator and validator for the intersect example. (sseth)
TEZ-1114. Fix encrypted shuffle. Contributed by Rajesh Balamohan.
TEZ-1128. OnFileUnorderedPartitionedKVOutput does not handle partitioning correctly with the MRPartitioner. (sseth)
TEZ-1121. Clean up avro dependencies. (hitesh)
TEZ-1111. TestMRHelpers fails if HADOOP_COMMON_HOME is defined in the shell env. ( Mohammad Kamrul Islam via hitesh)
TEZ-1099. Minor documentation improvement and Eclipse direct import friendlyness. Contributed by Thiruvalluvan M. G.
TEZ-1112. MROutput committer should be initialized from initialized OutputFormat. Contributed by Rohini Palaniswamy.
TEZ-1102. Abstract out connection management logic in shuffle code. Contributed by Rajesh Balamohan.
TEZ-1088. Flaky Test: TestFaultTolerance.testInputFailureCausesRerunAttemptWithinMaxAttemptSuccess (Tassapol Athiapinya via bikas)
TEZ-1105. Fix docs to ensure users are aware of adding "*" for HADOOP_CLASSPATH. (hitesh)
TEZ-1091. Respect keepAlive when shutting down Fetchers. Contributed by Rajesh Balamohan.
TEZ-1093. Add an example for OnFileUnorderedPartitionedOutput. (sseth)
TEZ-661. Add an implementation for a non sorted, partitioned, key-value output. (sseth)
TEZ-1002. Generate Container Stop history events. Contributed by Gopal V.
TEZ-1085. Leave env values unchanged if they aren't set on the client. Contributed by Rohini Palaniswamy.
TEZ-1082. Fix the mechanism used by the Fetcher to check for an open connection when draining the error stream. Contributed by Rajesh Balamohan
TEZ-886. Add @Nullable annotation at API level (Tsuyoshi OZAWA via bikas)
TEZ-802. Determination of Task Placement for 1-1 Edges (bikas)
TEZ-1079. Make tez example jobs use the ToolRunner framework (Devaraj K via bikas)
TEZ-1087. ShuffleManager fails with IllegalStateException (Cheolsoo Park via bikas)
TEZ-1074. Reduce the frequency at which counters are sent from the task to the AM to reduce AM CPU usage. Contributed by Rajesh Balamohan.
TEZ-1062. Create SimpleProcessor for processors that only need to implement the run method (Mohammad Kamrul Islam via bikas)
TEZ-1073. RLE fast-forward merge for IFile (gopalv)
TEZ-1023. Tez runtime configuration changes by users may not get propagated to jobs. Contributed by Rajesh Balamohan.
TEZ-698. Make it easy to create and configure MRInput/MROutput and other inputs/outputs (bikas)
TEZ-1018. VertexManagerPluginContext should enable assigning locality to scheduled tasks (bikas)
TEZ-1077. Add unit tests for SortedMergedGroupedInput. (sseth)
TEZ-1003. Add a MergedInput to combine multiple ShuffledMergedInputs. Contributed by Rohini Palaniswamy.
TEZ-873. Expose InputSplit via MRInputLegacy, and underlying splits via TezGroupedSplits. Contributed by Mohammad Kamrul Islam.
TEZ-737. DAG name should be unique within a Tez Session. (Mohammad Kamrul Islam via hitesh)
TEZ-919. Fix shutdown handling for Shuffle. (sseth)
TEZ-988. Enable KeepAlive in Tez Fetcher (Rajesh Balamohan via bikas)
TEZ-695. Create Abstract class for Input/Processor/Output (Mohammad Kamrul Islam via bikas)
Revert "TEZ-695. Create Abstract class for Input/Processor/Output (Mohammad Kamrul Islam via bikas)"
TEZ-695. Create Abstract class for Input/Processor/Output (Mohammad Kamrul Islam via bikas)
TEZ-708. Add a LocalTaskScheduler for use in Local mode. Contributed by Jonathan Eagles.
TEZ-700. Helper API's to monitor a DAG to completion (Mohammad Kamrul Islam via bikas)
TEZ-1053. Refactor: Pass TaskLocationHint directly to the Scheduling logic (bikas)
TEZ-1049. Refactor - LocationHint need not be passed into TaskAttemptImpl's constructor (bikas)