Class cascading.BasicPipesPlatformTest

16

tests

0

failures

0

ignored

14.884s

duration

100%

successful

Tests

Test Duration Result
testCoGroup 1.392s passed
testCoGroupRelativeUnknown 1.159s passed
testCount 1.109s passed
testCount2 0.608s passed
testCount3 0.601s passed
testCount4 0.572s passed
testFilter 0.405s passed
testFilterAll 0.451s passed
testReplace 0.430s passed
testSimple 0.571s passed
testSimpleChain 4.424s passed
testSimpleRelative 0.667s passed
testSimpleRelativeUnknown 0.623s passed
testSimpleResult 0.843s passed
testSwap 0.522s passed
testUnGroup 0.507s passed

Standard output

2016-05-17 21:52:27,423 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-17 21:52:27,454 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.BasicPipesPlatformTest, with platform: hadoop2-tez
2016-05-17 21:52:27,490 INFO  tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-17 21:52:27,850 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-17 21:52:28,206 INFO  platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides: 
2016-05-17 21:52:28,426 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:28,429 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:28,544 INFO  property.AppProps (AppProps.java:getAppID(177)) - using app.id: A74499D90A2B4DBF98BC2F2E01DD4D17
2016-05-17 21:52:28,738 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.089
2016-05-17 21:52:28,773 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.125
2016-05-17 21:52:28,774 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:28,775 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:28,776 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:28,926 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:29,124 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-17 21:52:29,397 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-17 21:52:29,400 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:29,401 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:29,402 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-17 21:52:29,403 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:29,403 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:29,403 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:29,405 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:29,405 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:29,428 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:29,602 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:29,610 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:29,610 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:29,650 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001 doesn't exist and is created
2016-05-17 21:52:29,711 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:29,711 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:29,713 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001
2016-05-17 21:52:29,758 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521949609_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:29,810 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:29,812 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-17 21:52:29,812 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-17 21:52:29,812 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:29,852 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:29,884 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:29,888 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:29,892 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001/localmode-log-dir
2016-05-17 21:52:29,892 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001/localmode-log-dir/history.txt.appattempt_1463521949609_0001_000000, maxErrors=10
2016-05-17 21:52:29,893 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:29,894 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521949609_0001_000000, appSubmitTime=1463521949738, launchTime=1463521949757
2016-05-17 21:52:29,900 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:29,900 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:29,902 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:29,905 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:29,913 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 21:52:29,913 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 21:52:29,913 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:29,960 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47089
2016-05-17 21:52:29,999 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:30,013 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 21:52:30,014 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 21:52:30,014 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:30,071 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:30,072 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521949609_0001_000000, startTime=1463521950071
2016-05-17 21:52:30,072 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:30,114 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:30,130 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:30,130 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463521949609_0001, dagName=[231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:30,335 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:30,432 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521949609_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001/localmode-log-dir/dag_1463521949609_0001_1.dot
2016-05-17 21:52:30,437 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001/localmode-log-dir/dag_1463521949609_0001_1-tez-dag.pb.txt
2016-05-17 21:52:30,534 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521949609_0001_1
2016-05-17 21:52:30,535 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:30,535 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:30 Running Dag: dag_1463521949609_0001_1
2016-05-17 21:52:30,564 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:30,564 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521949609_0001_1, submitTime=1463521950335
2016-05-17 21:52:30,593 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:30,600 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:30,602 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:30,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521949609_0001_1, initTime=1463521950565
2016-05-17 21:52:30,603 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521949609_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:30,605 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:30,605 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_STARTED]: dagID=dag_1463521949609_0001_1, startTime=1463521950604
2016-05-17 21:52:30,605 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:30,607 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521949609_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:30,608 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BC12AAA767694F8E8C505DF617C89107 : {3C2FF4BCEB3745B0AE2241B89B16E362={InputName=3C2FF4BCEB3745B0AE2241B89B16E362}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:30,610 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3C2FF4BCEB3745B0AE2241B89B16E362, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:30,610 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,615 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,615 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,616 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]. Starting root input initializers: 1
2016-05-17 21:52:30,617 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463521949609_0001, dagName=[231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:30,631 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3C2FF4BCEB3745B0AE2241B89B16E362 on vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,633 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:30,636 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:30,695 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521949609_0001, with dag id: dag_1463521949609_0001_1
2016-05-17 21:52:30,700 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:30,700 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:30,700 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:30,703 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3C2FF4BCEB3745B0AE2241B89B16E362 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:30,704 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:30,705 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:30,719 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:30,722 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:30,722 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, vertexId=vertex_1463521949609_0001_1_01, initRequestedTime=1463521950634, initedTime=1463521950721, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:30,723 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:30,731 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,755 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:30,756 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,763 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,763 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,764 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=27DB08F568EF4E09B6365C456C48AFD3, vertexId=vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:30,767 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:30,769 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3C2FF4BCEB3745B0AE2241B89B16E362 on vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,772 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=27DB08F568EF4E09B6365C456C48AFD3, vertexId=vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,790 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=27DB08F568EF4E09B6365C456C48AFD3, vertexName=E959E740B0D74FE8951993F5CB9EF4D4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:30,801 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=27DB08F568EF4E09B6365C456C48AFD3, vertexId=vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,805 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:30,805 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, vertexId=vertex_1463521949609_0001_1_02, initRequestedTime=1463521950731, initedTime=1463521950805, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:30,805 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:30,810 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:30,810 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3C2FF4BCEB3745B0AE2241B89B16E362=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:30,811 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] parallelism set to 1
2016-05-17 21:52:30,811 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:30,821 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521949609_0001_1_00, eventCount=1
2016-05-17 21:52:30,822 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:30,822 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BC12AAA767694F8E8C505DF617C89107, vertexId=vertex_1463521949609_0001_1_00, initRequestedTime=1463521950608, initedTime=1463521950822, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:30,823 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] with distanceFromRoot: 0
2016-05-17 21:52:30,823 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:30,832 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:30,833 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,835 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:30,835 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521949609_0001_1_00, startRequestedTime=1463521950831, startedTime=1463521950831
2016-05-17 21:52:30,837 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:30,864 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:30,865 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_STARTED]: vertexName=BC12AAA767694F8E8C505DF617C89107, taskId=task_1463521949609_0001_1_00_000000, scheduledTime=1463521950864, launchTime=1463521950864
2016-05-17 21:52:30,865 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:30,866 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521949609_0001_1_00 for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] numStartedSources: 1 numSources: 1
2016-05-17 21:52:30,866 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] with distanceFromRoot: 1
2016-05-17 21:52:30,868 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,872 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BC12AAA767694F8E8C505DF617C89107 in vertex: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:30,874 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BD9CBF0804B84E23A67381209AA8AA79 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:30,875 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:30,875 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521949609_0001_1_01, startRequestedTime=1463521950868, startedTime=1463521950868
2016-05-17 21:52:30,875 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:30,877 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: BC12AAA767694F8E8C505DF617C89107, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3C2FF4BCEB3745B0AE2241B89B16E362, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:30,879 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:30,879 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521949609_0001_1_01 for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] numStartedSources: 1 numSources: 1
2016-05-17 21:52:30,880 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] with distanceFromRoot: 2
2016-05-17 21:52:30,880 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:30,883 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BD9CBF0804B84E23A67381209AA8AA79 in vertex: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:30,883 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E959E740B0D74FE8951993F5CB9EF4D4 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:30,883 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521949609_0001_1_02, startRequestedTime=1463521950883, startedTime=1463521950883
2016-05-17 21:52:30,884 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:30,887 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:30,925 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:30,930 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:30,930 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:30,934 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521949609_0001_00_000001
2016-05-17 21:52:30,934 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:30,942 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521949609_0001_00_000001
2016-05-17 21:52:30,942 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:30,947 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:30,948 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:30,948 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521949609_0001_00_000001, launchTime=1463521950947
2016-05-17 21:52:30,951 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000001
2016-05-17 21:52:30,956 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521949609_0001_1_00_000000_0] to container: [container_1463521949609_0001_00_000001]
2016-05-17 21:52:30,956 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:30,958 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521949609_0001_00_000001 given task: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:30,958 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:30 Starting to run new task attempt: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:30,959 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:30,960 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:30,960 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521949609_0001_1_00_000000_0] started. Is using containerId: [container_1463521949609_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:30,961 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:30,962 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:30,962 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:30,962 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:30,963 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BC12AAA767694F8E8C505DF617C89107, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0, startTime=1463521950958, containerId=container_1463521949609_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521949609_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:30,964 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:30,964 INFO  counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters:  GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-17 21:52:30,965 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:30,968 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:30,989 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:30,990 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: BC12AAA767694F8E8C505DF617C89107, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3C2FF4BCEB3745B0AE2241B89B16E362, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:30,993 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:31,000 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,000 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:31,001 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,004 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521949609_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:31,008 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3C2FF4BCEB3745B0AE2241B89B16E362, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:31,009 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,010 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:31,010 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:31,011 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,016 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:31,017 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,026 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,037 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:31,038 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:31,039 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,040 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,043 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:31,043 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,043 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,053 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BC12AAA767694F8E8C505DF617C89107, ordinal: 0
2016-05-17 21:52:31,054 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC12AAA767694F8E8C505DF617C89107, mem on start (mb), free: 236, total: 281, max: 672
2016-05-17 21:52:31,055 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,055 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:31,055 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:31,056 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:31,057 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:31,061 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:31,062 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:31,062 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:31,063 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:31,063 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:31,064 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3C2FF4BCEB3745B0AE2241B89B16E362, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:31,064 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BD9CBF0804B84E23A67381209AA8AA79, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:31,064 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:31,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3C2FF4BCEB3745B0AE2241B89B16E362 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:31,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:31,067 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:31,067 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,068 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:31,068 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,068 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:31,071 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,072 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:31,083 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@49c9ccdb
2016-05-17 21:52:31,094 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@49c9ccdb
2016-05-17 21:52:31,095 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:31,096 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:31,111 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:31,153 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,153 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: D095011A42D942A78C13526AE8171DF1
2016-05-17 21:52:31,154 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] D095011A42D942A78C13526AE8171DF1
2016-05-17 21:52:31,200 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6742eba7; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@5a7d55e7; comparator=cascading.tuple.hadoop.util.TupleComparator@746ab4a7; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:31,205 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:31,208 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:31,209 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:31,334 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:31,334 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:31,334 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:31,334 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:31,335 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:31,337 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:31,337 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:31,339 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BC12AAA767694F8E8C505DF617C89107, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:52:31,348 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,348 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC12AAA767694F8E8C505DF617C89107, mem on close (mb), free: 123, total: 281, max: 672
2016-05-17 21:52:31,349 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:31,349 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:31,349 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 21:52:31,350 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 21:52:31,366 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:31 Completed running task attempt: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,389 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:31,390 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521949609_0001_1_00, eventCount=1
2016-05-17 21:52:31,392 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521951340, Process_Duration=8, Process_End_Time=1463521951348, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 21:52:31,393 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:31,393 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:31,394 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,394 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000001
2016-05-17 21:52:31,397 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BC12AAA767694F8E8C505DF617C89107, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0, startTime=1463521950958, finishTime=1463521951390, timeTaken=432, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951340, Process_Duration=8, Process_End_Time=1463521951348, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:31,398 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:31,398 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:31,402 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,404 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:31,408 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_FINISHED]: vertexName=BC12AAA767694F8E8C505DF617C89107, taskId=task_1463521949609_0001_1_00_000000, startTime=1463521950958, finishTime=1463521951403, timeTaken=445, status=SUCCEEDED, successfulAttemptID=attempt_1463521949609_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951340, Process_Duration=8, Process_End_Time=1463521951348, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:31,409 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:31,411 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:31,412 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] : 1
2016-05-17 21:52:31,412 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:31,422 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:31,427 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_FINISHED]: vertexName=BC12AAA767694F8E8C505DF617C89107, vertexId=vertex_1463521949609_0001_1_00, initRequestedTime=1463521950608, initedTime=1463521950822, startRequestedTime=1463521950831, startedTime=1463521950831, finishTime=1463521951412, timeTaken=581, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951340, Process_Duration=8, Process_End_Time=1463521951348, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463521950958, firstTasksToStart=[ task_1463521949609_0001_1_00_000000 ], lastTaskFinishTime=1463521951390, lastTasksToFinish=[ task_1463521949609_0001_1_00_000000 ], minTaskDuration=432, maxTaskDuration=432, avgTaskDuration=432.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521949609_0001_1_00_000000 ], longestDurationTasks=[ task_1463521949609_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:31,427 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:31,428 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521949609_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:31,429 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000001: 34 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:31,429 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521949609_0001_00_000001, Exiting
2016-05-17 21:52:31,430 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521949609_0001_00_000001
2016-05-17 21:52:31,430 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521949609_0001_00_000001
2016-05-17 21:52:31,431 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:31,431 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521949609_0001_00_000001, stoppedTime=1463521951431, exitStatus=0
2016-05-17 21:52:31,433 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:31,434 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] attempt: attempt_1463521949609_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:31,435 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521949609_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:31,436 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521949609_0001_00_000001 completed successfully
2016-05-17 21:52:31,436 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,438 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:31,441 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:31,441 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:31,441 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521949609_0001_00_000001
2016-05-17 21:52:31,443 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:31,443 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_STARTED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, taskId=task_1463521949609_0001_1_01_000000, scheduledTime=1463521951443, launchTime=1463521951443
2016-05-17 21:52:31,443 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:31,445 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:31,445 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: BD9CBF0804B84E23A67381209AA8AA79, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BC12AAA767694F8E8C505DF617C89107, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E959E740B0D74FE8951993F5CB9EF4D4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:31,446 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:31,446 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:31,447 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521949609_0001_00_000002
2016-05-17 21:52:31,447 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:31,452 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521949609_0001_00_000002
2016-05-17 21:52:31,453 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:31,456 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:31,457 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:31,457 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521949609_0001_00_000002, launchTime=1463521951456
2016-05-17 21:52:31,459 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000002
2016-05-17 21:52:31,460 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521949609_0001_1_01_000000_0] to container: [container_1463521949609_0001_00_000002]
2016-05-17 21:52:31,461 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:31,461 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521949609_0001_1_01_000000_0] started. Is using containerId: [container_1463521949609_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:31,462 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:31,462 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0, startTime=1463521951461, containerId=container_1463521949609_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521949609_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:31,462 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:31,462 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:31,463 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521949609_0001_00_000002 given task: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,463 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31 Starting to run new task attempt: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,464 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,464 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,465 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:31,465 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:31,465 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:31,465 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:31,466 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:31,466 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: BD9CBF0804B84E23A67381209AA8AA79, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BC12AAA767694F8E8C505DF617C89107, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E959E740B0D74FE8951993F5CB9EF4D4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:31,467 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:31,467 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,470 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:31,470 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,472 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BC12AAA767694F8E8C505DF617C89107, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:31,472 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:31,474 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,474 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E959E740B0D74FE8951993F5CB9EF4D4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:31,475 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:31,475 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:31,480 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:31,483 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,485 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:31,486 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:31,487 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:31,496 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BD9CBF0804B84E23A67381209AA8AA79, ordinal: 1
2016-05-17 21:52:31,496 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD9CBF0804B84E23A67381209AA8AA79, mem on start (mb), free: 113, total: 281, max: 672
2016-05-17 21:52:31,496 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,496 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:31,497 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:31,501 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521949609_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:31,507 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:31,507 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:31,507 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,509 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,509 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:31,510 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:31,510 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:31,512 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:31,512 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:31,512 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 739036352, TotalRequestedScaled: 3.69518176E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:1.05
2016-05-17 21:52:31,513 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E959E740B0D74FE8951993F5CB9EF4D4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=66985083
2016-05-17 21:52:31,513 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BC12AAA767694F8E8C505DF617C89107, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=405125774
2016-05-17 21:52:31,513 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:31,513 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BC12AAA767694F8E8C505DF617C89107 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:31,513 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:31,513 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:31,514 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,525 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:31,527 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:31,531 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BC12AAA767694F8E8C505DF617C89107 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:31,534 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=405125774. Updated to: ShuffleMem=405125774, postMergeMem: 0
2016-05-17 21:52:31,535 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=405125774, maxSingleShuffleLimit=101281440, mergeThreshold=364613184, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:31,538 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:31,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:31,544 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,548 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521949609_0001_1_00_000000_0_10003, runDuration: 343922, ]
2016-05-17 21:52:31,550 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:31,558 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: D095011A42D942A78C13526AE8171DF1
2016-05-17 21:52:31,558 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: D0EC84A6300E4617A6DFD1D38B5DC7A4
2016-05-17 21:52:31,559 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521949609_0001_1_00_000000_0_10003] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463521951559, TimeTaken=5, Rate=0.04 MB/s
2016-05-17 21:52:31,559 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,560 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:31,560 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BC12AAA767694F8E8C505DF617C89107] #1 in 7ms
2016-05-17 21:52:31,560 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] D0EC84A6300E4617A6DFD1D38B5DC7A4
2016-05-17 21:52:31,561 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BC12AAA767694F8E8C505DF617C89107] #1, status:false, isInterrupted:false
2016-05-17 21:52:31,565 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:31,568 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 21:52:31,569 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:31,570 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@58f03ef7; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@4b5f7bf; comparator=cascading.tuple.hadoop.util.TupleComparator@6841c69c; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:31,570 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:31,571 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:31,572 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:31,574 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:31,642 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 16515068(66060272)
2016-05-17 21:52:31,642 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 63
2016-05-17 21:52:31,643 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 52848232
2016-05-17 21:52:31,644 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 66060288
2016-05-17 21:52:31,644 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 16515068; length = 4128768
2016-05-17 21:52:31,646 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] D095011A42D942A78C13526AE8171DF1, for 1 inputs
2016-05-17 21:52:31,648 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 21:52:31,648 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,648 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BD9CBF0804B84E23A67381209AA8AA79, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:52:31,648 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:31,653 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5e73a225; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@799a5b94
2016-05-17 21:52:31,662 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,662 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD9CBF0804B84E23A67381209AA8AA79, mem on close (mb), free: 105, total: 345, max: 672
2016-05-17 21:52:31,662 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,664 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:31,665 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:31,665 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:31,666 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:31,666 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:31,667 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:31,667 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:31,667 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 184; bufvoid = 66060288
2016-05-17 21:52:31,668 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 16515068(66060272); kvend = 16515040(66060160); length = 29/4128768
2016-05-17 21:52:31,671 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:31 Completed running task attempt: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,676 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:31,677 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521949609_0001_1_01, eventCount=1
2016-05-17 21:52:31,678 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:31,680 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 49 [[File System Counters FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=121, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521951648, Process_Duration=14, Process_End_Time=1463521951662, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=1]]
2016-05-17 21:52:31,681 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:31,681 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,681 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0, startTime=1463521951461, finishTime=1463521951678, timeTaken=217, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=121, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951648, Process_Duration=14, Process_End_Time=1463521951662, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=1
2016-05-17 21:52:31,682 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:31,683 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,683 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:31,682 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000002
2016-05-17 21:52:31,686 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_FINISHED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, taskId=task_1463521949609_0001_1_01_000000, startTime=1463521951461, finishTime=1463521951683, timeTaken=222, status=SUCCEEDED, successfulAttemptID=attempt_1463521949609_0001_1_01_000000_0, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=121, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951648, Process_Duration=14, Process_End_Time=1463521951662, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=1
2016-05-17 21:52:31,686 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:31,687 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] : 1
2016-05-17 21:52:31,687 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:31,684 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:31,692 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:31,694 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_FINISHED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, vertexId=vertex_1463521949609_0001_1_01, initRequestedTime=1463521950634, initedTime=1463521950721, startRequestedTime=1463521950868, startedTime=1463521950868, finishTime=1463521951688, timeTaken=820, status=SUCCEEDED, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=121, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951648, Process_Duration=14, Process_End_Time=1463521951662, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=1, vertexStats=firstTaskStartTime=1463521951461, firstTasksToStart=[ task_1463521949609_0001_1_01_000000 ], lastTaskFinishTime=1463521951678, lastTasksToFinish=[ task_1463521949609_0001_1_01_000000 ], minTaskDuration=217, maxTaskDuration=217, avgTaskDuration=217.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521949609_0001_1_01_000000 ], longestDurationTasks=[ task_1463521949609_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:31,695 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:31,696 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:31,696 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:31,696 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521949609_0001_00_000002, stoppedTime=1463521951696, exitStatus=0
2016-05-17 21:52:31,697 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:31,697 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] attempt: attempt_1463521949609_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:31,697 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:31,698 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:31,698 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:31,698 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521949609_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:52:31,699 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_STARTED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, taskId=task_1463521949609_0001_1_02_000000, scheduledTime=1463521951698, launchTime=1463521951698
2016-05-17 21:52:31,699 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:31,699 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:31,699 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:31,700 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:31,700 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: E959E740B0D74FE8951993F5CB9EF4D4, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=27DB08F568EF4E09B6365C456C48AFD3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:31,700 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:31,700 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:31,701 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521949609_0001_00_000003
2016-05-17 21:52:31,701 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:31,703 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521949609_0001_00_000003
2016-05-17 21:52:31,703 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - TezTask metrics system already initialized!
2016-05-17 21:52:31,704 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:31,704 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521949609_0001_00_000003, launchTime=1463521951704
2016-05-17 21:52:31,704 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521949609_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:31,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:31,705 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000003
2016-05-17 21:52:31,705 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:31,705 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521949609_0001_00_000002, Exiting
2016-05-17 21:52:31,705 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521949609_0001_00_000002
2016-05-17 21:52:31,705 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521949609_0001_00_000002
2016-05-17 21:52:31,706 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521949609_0001_1_02_000000_0] to container: [container_1463521949609_0001_00_000003]
2016-05-17 21:52:31,707 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521949609_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:31,707 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:31,707 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521949609_0001_00_000002 completed successfully
2016-05-17 21:52:31,708 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:31,708 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521949609_0001_00_000003 given task: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,709 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521949609_0001_1_02_000000_0] started. Is using containerId: [container_1463521949609_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 21:52:31,709 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,710 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:31 Starting to run new task attempt: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,710 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,710 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,711 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:31,711 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:31,711 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:31,711 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:31,711 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:31,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: E959E740B0D74FE8951993F5CB9EF4D4, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=27DB08F568EF4E09B6365C456C48AFD3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:31,711 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:31,710 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0, startTime=1463521951709, containerId=container_1463521949609_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521949609_0001_00_000003/teamcity, completedLogs=
2016-05-17 21:52:31,712 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:31,713 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,713 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:31,713 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,713 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521949609_0001_1_02_000000_0 sent events: (0-1)
2016-05-17 21:52:31,713 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:31,714 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:31,714 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=27DB08F568EF4E09B6365C456C48AFD3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:31,714 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:31,714 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,715 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:31,714 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,715 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 27DB08F568EF4E09B6365C456C48AFD3
2016-05-17 21:52:31,715 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:31,715 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 27DB08F568EF4E09B6365C456C48AFD3
2016-05-17 21:52:31,716 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:31,716 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:31,716 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,716 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,727 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:31,741 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:31,741 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 27DB08F568EF4E09B6365C456C48AFD3
2016-05-17 21:52:31,744 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E959E740B0D74FE8951993F5CB9EF4D4, ordinal: 2
2016-05-17 21:52:31,744 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E959E740B0D74FE8951993F5CB9EF4D4, mem on start (mb), free: 98, total: 345, max: 672
2016-05-17 21:52:31,744 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,744 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:31,744 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:31,744 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:31,745 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:31,745 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:31,745 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:31,746 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:31,746 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:31,746 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:31,747 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 27DB08F568EF4E09B6365C456C48AFD3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:31,747 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BD9CBF0804B84E23A67381209AA8AA79, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:31,747 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:31,747 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BD9CBF0804B84E23A67381209AA8AA79 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:31,748 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,748 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:31,749 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:31,749 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:31,750 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:31,750 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BD9CBF0804B84E23A67381209AA8AA79 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:31,750 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:31,750 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:31,751 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:31,751 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,753 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:31,754 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,754 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521949609_0001_1_01_000000_0_10006, runDuration: 198687, ]
2016-05-17 21:52:31,759 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521949609_0001_1_01_000000_0_10006] to DISK_DIRECT, CompressedSize=210, DecompressedSize=206,EndTime=1463521951759, TimeTaken=2, Rate=0.10 MB/s
2016-05-17 21:52:31,759 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,759 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:31,759 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:31,760 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BD9CBF0804B84E23A67381209AA8AA79] #2 in 5ms
2016-05-17 21:52:31,761 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BD9CBF0804B84E23A67381209AA8AA79] #2, status:false, isInterrupted:false
2016-05-17 21:52:31,762 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:31,762 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: D0EC84A6300E4617A6DFD1D38B5DC7A4
2016-05-17 21:52:31,763 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"], id: 27DB08F568EF4E09B6365C456C48AFD3
2016-05-17 21:52:31,764 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-17 21:52:31,764 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 210 bytes from disk
2016-05-17 21:52:31,764 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:31,764 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:31,765 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2016-05-17 21:52:31,765 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,765 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] D0EC84A6300E4617A6DFD1D38B5DC7A4, for 1 inputs
2016-05-17 21:52:31,766 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:31,767 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E959E740B0D74FE8951993F5CB9EF4D4, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:31,772 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4ae22b2; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@4467d684
2016-05-17 21:52:31,774 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:31,775 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:31,775 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,776 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521949609_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 21:52:31,777 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,777 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E959E740B0D74FE8951993F5CB9EF4D4, mem on close (mb), free: 95, total: 345, max: 672
2016-05-17 21:52:31,777 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,778 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:31,779 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:31,779 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:31,779 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:31,780 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:31 Completed running task attempt: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,781 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:31,783 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:31,784 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521951767, Process_Duration=10, Process_End_Time=1463521951777, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 21:52:31,785 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:31,786 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,786 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000003
2016-05-17 21:52:31,786 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0, startTime=1463521951709, finishTime=1463521951783, timeTaken=74, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951767, Process_Duration=10, Process_End_Time=1463521951777, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:31,787 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:31,788 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,788 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:31,789 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:31,789 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_FINISHED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, taskId=task_1463521949609_0001_1_02_000000, startTime=1463521951709, finishTime=1463521951788, timeTaken=79, status=SUCCEEDED, successfulAttemptID=attempt_1463521949609_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951767, Process_Duration=10, Process_End_Time=1463521951777, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:31,790 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:31,790 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] : 1
2016-05-17 21:52:31,790 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:31,792 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_FINISHED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, vertexId=vertex_1463521949609_0001_1_02, initRequestedTime=1463521950731, initedTime=1463521950805, startRequestedTime=1463521950883, startedTime=1463521950883, finishTime=1463521951790, timeTaken=907, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951767, Process_Duration=10, Process_End_Time=1463521951777, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463521951709, firstTasksToStart=[ task_1463521949609_0001_1_02_000000 ], lastTaskFinishTime=1463521951783, lastTasksToFinish=[ task_1463521949609_0001_1_02_000000 ], minTaskDuration=74, maxTaskDuration=74, avgTaskDuration=74.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521949609_0001_1_02_000000 ], longestDurationTasks=[ task_1463521949609_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:31,792 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:31,792 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:31,793 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521949609_0001_00_000003, stoppedTime=1463521951792, exitStatus=0
2016-05-17 21:52:31,793 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:31,794 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:31,795 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:31,795 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521949609_0001_1
2016-05-17 21:52:31,796 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:31,799 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521949609_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 21:52:31,799 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000003: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:31,800 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521949609_0001_00_000003, Exiting
2016-05-17 21:52:31,800 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521949609_0001_00_000003
2016-05-17 21:52:31,800 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521949609_0001_00_000003
2016-05-17 21:52:31,800 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521949609_0001_1
2016-05-17 21:52:31,800 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521949609_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:31,801 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521949609_0001_00_000003 completed successfully
2016-05-17 21:52:31,800 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 27DB08F568EF4E09B6365C456C48AFD3 for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:31,812 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:31,813 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:31,820 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521949609_0001_1, queueSize=0
2016-05-17 21:52:31,821 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:31,823 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521949609_0001_1, startTime=1463521950604, finishTime=1463521951813, timeTaken=1209, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2716, FILE_BYTES_WRITTEN=1138, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=14, REDUCE_INPUT_RECORDS=18, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=36, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1018691584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=26, OUTPUT_BYTES=394, OUTPUT_BYTES_WITH_OVERHEAD=406, OUTPUT_BYTES_PHYSICAL=414, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=414, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=414, SHUFFLE_BYTES_DECOMPRESSED=406, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=414, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=42, MERGE_PHASE_TIME=136, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=31, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390565854755, Process_Duration=32, Process_End_Time=4390565854787, Read_Duration=6, Tuples_Read=28, Tuples_Written=26, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 21:52:31,824 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521949609_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:31,824 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521949609_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:31,827 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521949609_0001_00_000003
2016-05-17 21:52:31,827 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:31 Completed Dag: dag_1463521949609_0001_1
2016-05-17 21:52:31,829 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521949609_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:31,829 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521949609_0001_1_post
2016-05-17 21:52:31,829 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,831 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:31,837 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463521949609_0001
2016-05-17 21:52:31,841 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:31,841 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:31,841 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:31,841 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:31,841 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:31,843 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:31,844 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:31,844 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:31,844 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:31,845 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:31,845 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:31,845 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:31,845 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:31,851 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001
2016-05-17 21:52:31,853 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain/_temporary
2016-05-17 21:52:31,853 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:02.451
2016-05-17 21:52:31,911 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-17 21:52:31,957 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:31,958 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:32,033 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 21:52:32,062 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.055
2016-05-17 21:52:32,063 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:32,063 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:32,064 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:32,089 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 21:52:32,106 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count
2016-05-17 21:52:32,221 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 21:52:32,222 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:32,222 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-17 21:52:32,222 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  parallel execution of steps is enabled: false
2016-05-17 21:52:32,223 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  executing total steps: 1
2016-05-17 21:52:32,223 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  allocating management threads: 1
2016-05-17 21:52:32,224 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...splatform/testCount/count
2016-05-17 21:52:32,224 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:32,245 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 21:52:32,250 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:32,250 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:32,251 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:32,262 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001 doesn't exist and is created
2016-05-17 21:52:32,285 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:32,285 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:32,286 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001
2016-05-17 21:52:32,290 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521952250_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:32,305 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:32,306 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:32,308 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:32,308 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:32,308 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001/localmode-log-dir
2016-05-17 21:52:32,309 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001/localmode-log-dir/history.txt.appattempt_1463521952250_0001_000000, maxErrors=10
2016-05-17 21:52:32,309 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:32,309 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521952250_0001_000000, appSubmitTime=1463521952290, launchTime=1463521952290
2016-05-17 21:52:32,317 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:32,319 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59387
2016-05-17 21:52:32,321 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:32,324 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:32,324 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:32,328 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:32,332 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521952250_0001_000000, startTime=1463521952331
2016-05-17 21:52:32,332 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:32,332 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:32,386 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:32,386 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:32,386 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463521952250_0001, dagName=[5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count
2016-05-17 21:52:32,413 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count
2016-05-17 21:52:32,419 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521952250_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001/localmode-log-dir/dag_1463521952250_0001_1.dot
2016-05-17 21:52:32,421 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001/localmode-log-dir/dag_1463521952250_0001_1-tez-dag.pb.txt
2016-05-17 21:52:32,463 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521952250_0001_1
2016-05-17 21:52:32,463 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,464 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count
2016-05-17 21:52:32 Running Dag: dag_1463521952250_0001_1
2016-05-17 21:52:32,484 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:32,484 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521952250_0001_1, submitTime=1463521952413
2016-05-17 21:52:32,486 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,487 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:32,488 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:32,488 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521952250_0001_1, initTime=1463521952485
2016-05-17 21:52:32,488 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521952250_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:32,490 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_STARTED]: dagID=dag_1463521952250_0001_1, startTime=1463521952489
2016-05-17 21:52:32,490 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:32,490 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521952250_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:32,490 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:32,491 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FFDB880A50474694B112BB25B58188B0 : {E8EAAE3498134BE0AF141663FC08543C={InputName=E8EAAE3498134BE0AF141663FC08543C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:32,491 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E8EAAE3498134BE0AF141663FC08543C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:32,491 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,491 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,491 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,492 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]. Starting root input initializers: 1
2016-05-17 21:52:32,493 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:32,493 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,494 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E8EAAE3498134BE0AF141663FC08543C on vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,496 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463521952250_0001, dagName=[5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count
2016-05-17 21:52:32,503 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:32,504 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,504 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,505 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,505 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3B2A087FB8BC40B4B51AA03410512F76, vertexId=vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:32,506 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3B2A087FB8BC40B4B51AA03410512F76, vertexId=vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,512 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:32,514 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E8EAAE3498134BE0AF141663FC08543C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:32,517 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:32,517 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:32,517 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3B2A087FB8BC40B4B51AA03410512F76, vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:32,518 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3B2A087FB8BC40B4B51AA03410512F76, vertexId=vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,521 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 21:52:32,524 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:32,524 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, vertexId=vertex_1463521952250_0001_1_01, initRequestedTime=1463521952493, initedTime=1463521952524, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:32,525 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:32,525 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E8EAAE3498134BE0AF141663FC08543C on vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,525 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:32,526 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E8EAAE3498134BE0AF141663FC08543C=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:32,528 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] parallelism set to 1
2016-05-17 21:52:32,528 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:32,529 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521952250_0001_1_00, eventCount=1
2016-05-17 21:52:32,530 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:32,530 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FFDB880A50474694B112BB25B58188B0, vertexId=vertex_1463521952250_0001_1_00, initRequestedTime=1463521952490, initedTime=1463521952530, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:32,532 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] with distanceFromRoot: 0
2016-05-17 21:52:32,532 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463521952250_0001, with dag id: dag_1463521952250_0001_1
2016-05-17 21:52:32,532 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:32,533 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,533 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,535 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:32,535 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521952250_0001_1_00, startRequestedTime=1463521952533, startedTime=1463521952533
2016-05-17 21:52:32,536 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:32,537 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:32,537 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_STARTED]: vertexName=FFDB880A50474694B112BB25B58188B0, taskId=task_1463521952250_0001_1_00_000000, scheduledTime=1463521952537, launchTime=1463521952537
2016-05-17 21:52:32,538 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:32,538 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521952250_0001_1_00 for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] numStartedSources: 1 numSources: 1
2016-05-17 21:52:32,540 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] with distanceFromRoot: 1
2016-05-17 21:52:32,540 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,553 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FFDB880A50474694B112BB25B58188B0 in vertex: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,554 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EA6F324C2A0443EE8B19EA89C639CEF3 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:32,554 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521952250_0001_1_01, startRequestedTime=1463521952543, startedTime=1463521952543
2016-05-17 21:52:32,554 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:32,554 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:32,558 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count, VertexName: FFDB880A50474694B112BB25B58188B0, VertexParallelism: 1, TaskAttemptID:attempt_1463521952250_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8EAAE3498134BE0AF141663FC08543C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA6F324C2A0443EE8B19EA89C639CEF3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:32,560 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:32,561 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:32,564 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:32,564 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:32,564 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:32,565 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521952250_0001_00_000001
2016-05-17 21:52:32,565 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:32,571 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521952250_0001_00_000001
2016-05-17 21:52:32,571 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:32,572 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:32,573 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:32,573 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521952250_0001_00_000001, launchTime=1463521952572
2016-05-17 21:52:32,575 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521952250_0001_00_000001
2016-05-17 21:52:32,578 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521952250_0001_1_00_000000_0] to container: [container_1463521952250_0001_00_000001]
2016-05-17 21:52:32,578 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:32,579 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521952250_0001_1_00_000000_0] started. Is using containerId: [container_1463521952250_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:32,579 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:32,579 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FFDB880A50474694B112BB25B58188B0, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0, startTime=1463521952578, containerId=container_1463521952250_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521952250_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:32,579 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:32,580 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:32,580 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521952250_0001_00_000001 given task: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,581 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521952250_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32 Starting to run new task attempt: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,581 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,581 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,582 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:32,582 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:32,582 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:32,582 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:32,582 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:32,582 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count, VertexName: FFDB880A50474694B112BB25B58188B0, VertexParallelism: 1, TaskAttemptID:attempt_1463521952250_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8EAAE3498134BE0AF141663FC08543C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA6F324C2A0443EE8B19EA89C639CEF3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:32,583 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:32,583 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,585 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:32,585 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,587 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E8EAAE3498134BE0AF141663FC08543C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:32,588 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:32,589 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,590 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EA6F324C2A0443EE8B19EA89C639CEF3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:32,591 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521952250_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:32,595 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:32,596 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,597 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:32,601 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:32,601 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,608 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:32,609 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,609 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,610 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FFDB880A50474694B112BB25B58188B0, ordinal: 0
2016-05-17 21:52:32,611 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FFDB880A50474694B112BB25B58188B0, mem on start (mb), free: 113, total: 345, max: 672
2016-05-17 21:52:32,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:32,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:32,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:32,611 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:32,612 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:32,612 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:32,612 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:32,613 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:32,613 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:32,613 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E8EAAE3498134BE0AF141663FC08543C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:32,614 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EA6F324C2A0443EE8B19EA89C639CEF3, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:32,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:32,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,615 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:32,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E8EAAE3498134BE0AF141663FC08543C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:32,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:32,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:32,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:32,617 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,623 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:32,625 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@43a08a6b
2016-05-17 21:52:32,625 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:32,627 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,627 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 0ABF8795FBD0485EA0C193EDB8452F9B
2016-05-17 21:52:32,627 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 0ABF8795FBD0485EA0C193EDB8452F9B
2016-05-17 21:52:32,630 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@43a08a6b
2016-05-17 21:52:32,632 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:32,632 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:32,632 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2fef37c0; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@23536d6; comparator=cascading.tuple.hadoop.util.TupleComparator@73a31cbb; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:32,637 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:32,638 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:32,638 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:32,760 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:32,761 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:32,761 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:32,761 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:32,762 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:32,762 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:32,763 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:32,763 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FFDB880A50474694B112BB25B58188B0, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:32,766 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,766 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FFDB880A50474694B112BB25B58188B0, mem on close (mb), free: 111, total: 445, max: 672
2016-05-17 21:52:32,766 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:32,767 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:32,767 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 21:52:32,767 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:52:32,772 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:32 Completed running task attempt: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,777 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:32,778 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521952250_0001_1_00, eventCount=1
2016-05-17 21:52:32,780 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:32,781 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FFDB880A50474694B112BB25B58188B0, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0, startTime=1463521952578, finishTime=1463521952779, timeTaken=201, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952763, Process_Duration=3, Process_End_Time=1463521952766, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:32,781 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:32,782 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,782 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:32,787 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_FINISHED]: vertexName=FFDB880A50474694B112BB25B58188B0, taskId=task_1463521952250_0001_1_00_000000, startTime=1463521952578, finishTime=1463521952782, timeTaken=204, status=SUCCEEDED, successfulAttemptID=attempt_1463521952250_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952763, Process_Duration=3, Process_End_Time=1463521952766, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:32,788 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:32,790 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] : 1
2016-05-17 21:52:32,787 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:32,786 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521952763, Process_Duration=3, Process_End_Time=1463521952766, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:52:32,790 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,791 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:32,792 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,793 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:32,793 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521952250_0001_00_000001
2016-05-17 21:52:32,794 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_FINISHED]: vertexName=FFDB880A50474694B112BB25B58188B0, vertexId=vertex_1463521952250_0001_1_00, initRequestedTime=1463521952490, initedTime=1463521952530, startRequestedTime=1463521952533, startedTime=1463521952533, finishTime=1463521952791, timeTaken=258, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952763, Process_Duration=3, Process_End_Time=1463521952766, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463521952578, firstTasksToStart=[ task_1463521952250_0001_1_00_000000 ], lastTaskFinishTime=1463521952779, lastTasksToFinish=[ task_1463521952250_0001_1_00_000000 ], minTaskDuration=201, maxTaskDuration=201, avgTaskDuration=201.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521952250_0001_1_00_000000 ], longestDurationTasks=[ task_1463521952250_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:32,795 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:32,795 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:32,795 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] attempt: attempt_1463521952250_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:32,795 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,796 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,796 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521952250_0001_00_000001, stoppedTime=1463521952796, exitStatus=0
2016-05-17 21:52:32,796 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:32,796 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:32,797 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:32,797 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:32,798 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_STARTED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, taskId=task_1463521952250_0001_1_01_000000, scheduledTime=1463521952797, launchTime=1463521952797
2016-05-17 21:52:32,798 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:32,798 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521952250_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:32,799 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,799 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:32,799 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count, VertexName: EA6F324C2A0443EE8B19EA89C639CEF3, VertexParallelism: 1, TaskAttemptID:attempt_1463521952250_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FFDB880A50474694B112BB25B58188B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3B2A087FB8BC40B4B51AA03410512F76, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:32,799 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:32,800 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:32,801 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521952250_0001_00_000002
2016-05-17 21:52:32,801 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:32,803 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521952250_0001_00_000002
2016-05-17 21:52:32,803 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:32,804 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:32,804 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521952250_0001_00_000002, launchTime=1463521952804
2016-05-17 21:52:32,804 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:32,805 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521952250_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:32,805 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521952250_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:32,805 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521952250_0001_00_000001, Exiting
2016-05-17 21:52:32,805 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521952250_0001_00_000001
2016-05-17 21:52:32,805 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521952250_0001_00_000001
2016-05-17 21:52:32,806 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521952250_0001_00_000002
2016-05-17 21:52:32,807 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521952250_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:32,807 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521952250_0001_00_000001 completed successfully
2016-05-17 21:52:32,807 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521952250_0001_1_01_000000_0] to container: [container_1463521952250_0001_00_000002]
2016-05-17 21:52:32,808 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:32,808 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:32,808 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521952250_0001_1_01_000000_0] started. Is using containerId: [container_1463521952250_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:32,809 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:32,809 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0, startTime=1463521952808, containerId=container_1463521952250_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521952250_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:32,809 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:32,810 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:32,808 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521952250_0001_00_000002 given task: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,810 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521952250_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32 Starting to run new task attempt: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,811 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,811 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,811 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:32,811 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:32,811 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:32,811 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:32,812 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:32,812 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count, VertexName: EA6F324C2A0443EE8B19EA89C639CEF3, VertexParallelism: 1, TaskAttemptID:attempt_1463521952250_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FFDB880A50474694B112BB25B58188B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3B2A087FB8BC40B4B51AA03410512F76, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:32,812 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:32,813 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,815 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:32,815 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,816 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FFDB880A50474694B112BB25B58188B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:32,816 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:32,816 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,816 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,823 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521952250_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:32,824 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:32,825 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3B2A087FB8BC40B4B51AA03410512F76, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:32,825 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:32,826 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3B2A087FB8BC40B4B51AA03410512F76
2016-05-17 21:52:32,826 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:32,826 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3B2A087FB8BC40B4B51AA03410512F76
2016-05-17 21:52:32,828 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:32,828 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:32,836 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,838 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,848 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:32,848 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3B2A087FB8BC40B4B51AA03410512F76
2016-05-17 21:52:32,849 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EA6F324C2A0443EE8B19EA89C639CEF3, ordinal: 1
2016-05-17 21:52:32,850 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA6F324C2A0443EE8B19EA89C639CEF3, mem on start (mb), free: 103, total: 445, max: 672
2016-05-17 21:52:32,850 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,850 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:32,850 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:32,850 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:32,850 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:32,850 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:32,850 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:32,851 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:32,851 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:32,851 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:32,852 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3B2A087FB8BC40B4B51AA03410512F76, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:32,852 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FFDB880A50474694B112BB25B58188B0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:32,852 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:32,852 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FFDB880A50474694B112BB25B58188B0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:32,853 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:32,853 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:32,853 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,854 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:32,855 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:32,855 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FFDB880A50474694B112BB25B58188B0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:32,857 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:32,857 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:32,857 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:32,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:32,859 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,862 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521952250_0001_1_00_000000_0_10012, runDuration: 178816, ]
2016-05-17 21:52:32,864 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521952250_0001_1_00_000000_0_10012] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463521952864, TimeTaken=2, Rate=0.22 MB/s
2016-05-17 21:52:32,864 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,864 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:32,865 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [FFDB880A50474694B112BB25B58188B0] #3 in 3ms
2016-05-17 21:52:32,866 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 21:52:32,866 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FFDB880A50474694B112BB25B58188B0] #3, status:false, isInterrupted:false
2016-05-17 21:52:32,866 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 0ABF8795FBD0485EA0C193EDB8452F9B
2016-05-17 21:52:32,867 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"], id: 3B2A087FB8BC40B4B51AA03410512F76
2016-05-17 21:52:32,867 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-17 21:52:32,867 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 0ABF8795FBD0485EA0C193EDB8452F9B, for 1 inputs
2016-05-17 21:52:32,868 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:32,868 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 21:52:32,869 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:32,869 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:32,869 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 21:52:32,870 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,870 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:32,870 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EA6F324C2A0443EE8B19EA89C639CEF3, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:52:32,872 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7e779b1b; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5e684a7a
2016-05-17 21:52:32,876 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:32,876 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:32,877 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,877 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521952250_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:32,879 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,879 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA6F324C2A0443EE8B19EA89C639CEF3, mem on close (mb), free: 100, total: 445, max: 672
2016-05-17 21:52:32,879 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,879 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:32,880 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:32,880 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:32,881 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:32,881 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:32 Completed running task attempt: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,883 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:32,884 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:32,885 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521952871, Process_Duration=7, Process_End_Time=1463521952878, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 21:52:32,886 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0, startTime=1463521952808, finishTime=1463521952884, timeTaken=76, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952871, Process_Duration=7, Process_End_Time=1463521952878, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:32,886 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:32,886 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:32,886 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,887 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:32,889 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_FINISHED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, taskId=task_1463521952250_0001_1_01_000000, startTime=1463521952808, finishTime=1463521952887, timeTaken=79, status=SUCCEEDED, successfulAttemptID=attempt_1463521952250_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952871, Process_Duration=7, Process_End_Time=1463521952878, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:32,889 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:32,889 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] : 1
2016-05-17 21:52:32,890 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,890 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:32,890 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,891 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521952250_0001_00_000002
2016-05-17 21:52:32,892 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:32,893 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_FINISHED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, vertexId=vertex_1463521952250_0001_1_01, initRequestedTime=1463521952493, initedTime=1463521952524, startRequestedTime=1463521952543, startedTime=1463521952543, finishTime=1463521952890, timeTaken=347, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952871, Process_Duration=7, Process_End_Time=1463521952878, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463521952808, firstTasksToStart=[ task_1463521952250_0001_1_01_000000 ], lastTaskFinishTime=1463521952884, lastTasksToFinish=[ task_1463521952250_0001_1_01_000000 ], minTaskDuration=76, maxTaskDuration=76, avgTaskDuration=76.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521952250_0001_1_01_000000 ], longestDurationTasks=[ task_1463521952250_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:32,893 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:32,897 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:32,897 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521952250_0001_00_000002, stoppedTime=1463521952897, exitStatus=0
2016-05-17 21:52:32,898 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:32,898 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:32,898 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:32,898 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521952250_0001_1
2016-05-17 21:52:32,899 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521952250_0001_1
2016-05-17 21:52:32,899 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3B2A087FB8BC40B4B51AA03410512F76 for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,901 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521952250_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:32,902 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521952250_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:32,902 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521952250_0001_00_000002, Exiting
2016-05-17 21:52:32,902 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521952250_0001_00_000002
2016-05-17 21:52:32,902 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521952250_0001_00_000002
2016-05-17 21:52:32,903 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521952250_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:32,903 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521952250_0001_00_000002 completed successfully
2016-05-17 21:52:32,913 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,916 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521952250_0001_1, queueSize=0
2016-05-17 21:52:32,916 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:32,916 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521952250_0001_1, startTime=1463521952489, finishTime=1463521952913, timeTaken=424, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=934281216, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043905634, Process_Duration=10, Process_End_Time=2927043905644, Read_Duration=4, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 21:52:32,916 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521952250_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:32,917 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521952250_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:32,917 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521952250_0001_00_000002
2016-05-17 21:52:32,918 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:32 Completed Dag: dag_1463521952250_0001_1
2016-05-17 21:52:32,920 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521952250_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:32,921 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521952250_0001_1_post
2016-05-17 21:52:32,921 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,921 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:32,926 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463521952250_0001
2016-05-17 21:52:32,929 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:32,929 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:32,929 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:32,929 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:32,929 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:32,930 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:32,930 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:32,930 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:32,931 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:32,931 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:32,931 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:32,932 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:32,932 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:32,936 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001
2016-05-17 21:52:32,937 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count/_temporary
2016-05-17 21:52:32,938 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  completed in: 00:00.717
2016-05-17 21:52:33,022 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount/count
2016-05-17 21:52:33,049 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:33,050 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:33,102 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-17 21:52:33,124 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-17 21:52:33,125 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:33,125 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:33,126 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:33,146 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:33,147 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-17 21:52:33,158 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:33,159 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:33,159 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-17 21:52:33,159 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:33,159 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:33,159 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:33,160 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33,160 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:33,177 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:33,182 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:33,182 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:33,183 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:33,196 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001 doesn't exist and is created
2016-05-17 21:52:33,215 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:33,216 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:33,216 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001
2016-05-17 21:52:33,221 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521953182_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:33,233 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:33,234 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:33,236 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:33,236 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:33,236 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001/localmode-log-dir
2016-05-17 21:52:33,236 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001/localmode-log-dir/history.txt.appattempt_1463521953182_0001_000000, maxErrors=10
2016-05-17 21:52:33,237 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:33,237 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521953182_0001_000000, appSubmitTime=1463521953220, launchTime=1463521953220
2016-05-17 21:52:33,261 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:33,265 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:33,268 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51682
2016-05-17 21:52:33,269 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:33,269 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:33,274 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:33,278 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521953182_0001_000000, startTime=1463521953278
2016-05-17 21:52:33,278 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:33,278 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:33,316 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:33,316 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:33,316 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463521953182_0001, dagName=[166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33,338 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33,339 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521953182_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001/localmode-log-dir/dag_1463521953182_0001_1.dot
2016-05-17 21:52:33,340 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001/localmode-log-dir/dag_1463521953182_0001_1-tez-dag.pb.txt
2016-05-17 21:52:33,366 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521953182_0001_1
2016-05-17 21:52:33,366 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,366 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33 Running Dag: dag_1463521953182_0001_1
2016-05-17 21:52:33,382 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:33,382 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521953182_0001_1, submitTime=1463521953338
2016-05-17 21:52:33,382 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AC05A3849AB84C8E941B50FCD8DFFAD1
2016-05-17 21:52:33,383 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:33,383 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:33,383 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521953182_0001_1, initTime=1463521953382
2016-05-17 21:52:33,383 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953182_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:33,385 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_STARTED]: dagID=dag_1463521953182_0001_1, startTime=1463521953385
2016-05-17 21:52:33,385 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:33,385 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953182_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:33,385 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AC05A3849AB84C8E941B50FCD8DFFAD1 : {66D86C6E329645E794BE88DC26DA2EE1={InputName=66D86C6E329645E794BE88DC26DA2EE1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:33,386 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 66D86C6E329645E794BE88DC26DA2EE1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:33,386 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,386 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,386 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,386 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]. Starting root input initializers: 1
2016-05-17 21:52:33,386 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:33,387 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 66D86C6E329645E794BE88DC26DA2EE1 on vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,387 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463521953182_0001, dagName=[166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33,387 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:33,389 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:33,391 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 66D86C6E329645E794BE88DC26DA2EE1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:33,392 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:33,392 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:33,395 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:33,395 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 66D86C6E329645E794BE88DC26DA2EE1 on vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,396 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:33,396 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {66D86C6E329645E794BE88DC26DA2EE1=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:33,396 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] parallelism set to 1
2016-05-17 21:52:33,396 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:33,397 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521953182_0001_1_00, eventCount=1
2016-05-17 21:52:33,397 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,397 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=558FAA5C8ABD4956B9A128A9A68E513F, vertexId=vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:33,397 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=558FAA5C8ABD4956B9A128A9A68E513F, vertexId=vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,399 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=558FAA5C8ABD4956B9A128A9A68E513F, vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:33,400 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=558FAA5C8ABD4956B9A128A9A68E513F, vertexId=vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,402 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, vertexId=vertex_1463521953182_0001_1_00, initRequestedTime=1463521953385, initedTime=1463521953402, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:33,402 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] with distanceFromRoot: 0
2016-05-17 21:52:33,402 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:33,402 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:33,403 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AC05A3849AB84C8E941B50FCD8DFFAD1
2016-05-17 21:52:33,403 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,404 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521953182_0001_1_00, startRequestedTime=1463521953403, startedTime=1463521953403
2016-05-17 21:52:33,404 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:33,404 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:33,405 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:33,405 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:TASK_STARTED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, taskId=task_1463521953182_0001_1_00_000000, scheduledTime=1463521953405, launchTime=1463521953405
2016-05-17 21:52:33,405 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953182_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:33,406 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,406 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap, VertexName: AC05A3849AB84C8E941B50FCD8DFFAD1, VertexParallelism: 1, TaskAttemptID:attempt_1463521953182_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=66D86C6E329645E794BE88DC26DA2EE1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=558FAA5C8ABD4956B9A128A9A68E513F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:33,407 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953182_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:33,407 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:33,408 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:33,409 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:33,409 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:33,409 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521953182_0001_00_000001
2016-05-17 21:52:33,409 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:33,411 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521953182_0001_00_000001
2016-05-17 21:52:33,412 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:33,413 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:33,413 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:33,413 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521953182_0001_00_000001, launchTime=1463521953413
2016-05-17 21:52:33,414 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521953182_0001_00_000001
2016-05-17 21:52:33,415 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521953182_0001_1_00_000000_0] to container: [container_1463521953182_0001_00_000001]
2016-05-17 21:52:33,415 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:33,415 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521953182_0001_1_00_000000_0] started. Is using containerId: [container_1463521953182_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:33,415 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0, startTime=1463521953415, containerId=container_1463521953182_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521953182_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:33,416 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:33,416 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953182_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:33,416 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953182_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:33,416 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521953182_0001_00_000001 given task: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,416 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521953182_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33 Starting to run new task attempt: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,417 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,417 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,417 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:33,417 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:33,417 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:33,417 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:33,418 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:33,418 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap, VertexName: AC05A3849AB84C8E941B50FCD8DFFAD1, VertexParallelism: 1, TaskAttemptID:attempt_1463521953182_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=66D86C6E329645E794BE88DC26DA2EE1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=558FAA5C8ABD4956B9A128A9A68E513F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:33,418 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:33,420 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521953182_0001, with dag id: dag_1463521953182_0001_1
2016-05-17 21:52:33,421 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,424 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:33,424 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,426 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521953182_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:33,427 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=66D86C6E329645E794BE88DC26DA2EE1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:33,428 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=558FAA5C8ABD4956B9A128A9A68E513F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:33,428 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:33,427 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,429 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:33,429 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 558FAA5C8ABD4956B9A128A9A68E513F
2016-05-17 21:52:33,429 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:33,430 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 558FAA5C8ABD4956B9A128A9A68E513F
2016-05-17 21:52:33,431 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,432 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:33,434 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,437 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:33,437 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,437 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,444 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AC05A3849AB84C8E941B50FCD8DFFAD1, ordinal: 0
2016-05-17 21:52:33,444 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC05A3849AB84C8E941B50FCD8DFFAD1, mem on start (mb), free: 241, total: 555, max: 672
2016-05-17 21:52:33,444 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:33,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 558FAA5C8ABD4956B9A128A9A68E513F
2016-05-17 21:52:33,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:33,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:33,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:33,446 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:33,446 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:33,446 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:33,447 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:33,447 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:33,447 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:33,447 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:33,448 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 558FAA5C8ABD4956B9A128A9A68E513F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:33,448 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 66D86C6E329645E794BE88DC26DA2EE1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:33,448 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:33,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 66D86C6E329645E794BE88DC26DA2EE1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:33,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,449 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:33,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:33,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:33,450 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:33,450 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,450 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:33,451 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@204e1f9c
2016-05-17 21:52:33,452 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@204e1f9c
2016-05-17 21:52:33,453 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:33,453 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:33,453 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:33,454 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,454 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"], id: 558FAA5C8ABD4956B9A128A9A68E513F
2016-05-17 21:52:33,455 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-17 21:52:33,455 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:33,455 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:33,456 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AC05A3849AB84C8E941B50FCD8DFFAD1, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:33,459 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:33,460 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:33,460 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,460 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521953182_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:33,461 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,461 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC05A3849AB84C8E941B50FCD8DFFAD1, mem on close (mb), free: 239, total: 555, max: 672
2016-05-17 21:52:33 Completed running task attempt: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,462 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:33,463 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:33,464 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0, startTime=1463521953415, finishTime=1463521953463, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:33,464 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953182_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:33,464 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:33,464 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,465 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:33,465 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:TASK_FINISHED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, taskId=task_1463521953182_0001_1_00_000000, startTime=1463521953415, finishTime=1463521953465, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1463521953182_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:33,465 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953182_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:33,466 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] : 1
2016-05-17 21:52:33,466 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,466 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-17 21:52:33,467 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:33,467 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,468 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:VERTEX_FINISHED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, vertexId=vertex_1463521953182_0001_1_00, initRequestedTime=1463521953385, initedTime=1463521953402, startRequestedTime=1463521953403, startedTime=1463521953403, finishTime=1463521953466, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463521953415, firstTasksToStart=[ task_1463521953182_0001_1_00_000000 ], lastTaskFinishTime=1463521953463, lastTasksToFinish=[ task_1463521953182_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521953182_0001_1_00_000000 ], longestDurationTasks=[ task_1463521953182_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:33,468 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:33,468 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:33,468 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521953182_0001_00_000001, stoppedTime=1463521953468, exitStatus=0
2016-05-17 21:52:33,469 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:33,469 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:33,468 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:33,470 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:33,468 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521953182_0001_00_000001
2016-05-17 21:52:33,469 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521953182_0001_1
2016-05-17 21:52:33,472 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521953182_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:33,472 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521953182_0001_1
2016-05-17 21:52:33,473 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 558FAA5C8ABD4956B9A128A9A68E513F for vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,472 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521953182_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:33,473 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521953182_0001_00_000001, Exiting
2016-05-17 21:52:33,473 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521953182_0001_00_000001
2016-05-17 21:52:33,473 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521953182_0001_00_000001
2016-05-17 21:52:33,476 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521953182_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:33,476 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521953182_0001_00_000001 completed successfully
2016-05-17 21:52:33,484 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521953182_0001_1, queueSize=0
2016-05-17 21:52:33,484 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:33,485 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521953182_0001_1, startTime=1463521953385, finishTime=1463521953483, timeTaken=98, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:33,485 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521953182_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:33,486 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953182_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:33,486 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521953182_0001_00_000001
2016-05-17 21:52:33,486 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:33 Completed Dag: dag_1463521953182_0001_1
2016-05-17 21:52:33,487 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521953182_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:33,487 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521953182_0001_1_post
2016-05-17 21:52:33,487 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,487 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:33,493 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463521953182_0001
2016-05-17 21:52:33,495 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:33,495 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:33,496 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:33,496 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:33,496 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:33,496 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:33,497 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:33,497 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:33,497 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:33,498 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:33,498 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:33,499 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:33,499 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:33,505 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001
2016-05-17 21:52:33,506 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap/_temporary
2016-05-17 21:52:33,506 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.347
2016-05-17 21:52:33,545 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-17 21:52:33,586 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:33,586 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:33,630 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 21:52:33,642 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-17 21:52:33,642 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:33,643 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:33,643 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:33,661 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt
2016-05-17 21:52:33,662 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-17 21:52:33,679 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:33,679 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 21:52:33,680 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-17 21:52:33,680 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:33,680 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:33,680 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:33,681 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33,682 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:33,696 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:33,701 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:33,701 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:33,701 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:33,712 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001 doesn't exist and is created
2016-05-17 21:52:33,734 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:33,734 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:33,735 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001
2016-05-17 21:52:33,739 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521953701_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:33,751 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:33,753 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:33,755 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:33,755 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:33,755 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001/localmode-log-dir
2016-05-17 21:52:33,755 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001/localmode-log-dir/history.txt.appattempt_1463521953701_0001_000000, maxErrors=10
2016-05-17 21:52:33,756 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:33,757 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521953701_0001_000000, appSubmitTime=1463521953738, launchTime=1463521953738
2016-05-17 21:52:33,763 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:33,764 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:33,766 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:33,767 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:33,770 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48724
2016-05-17 21:52:33,772 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:33,774 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521953701_0001_000000, startTime=1463521953774
2016-05-17 21:52:33,774 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:33,774 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:33,835 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:33,835 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:33,835 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463521953701_0001, dagName=[5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33,856 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33,858 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521953701_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001/localmode-log-dir/dag_1463521953701_0001_1.dot
2016-05-17 21:52:33,859 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001/localmode-log-dir/dag_1463521953701_0001_1-tez-dag.pb.txt
2016-05-17 21:52:33,882 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521953701_0001_1
2016-05-17 21:52:33,882 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,882 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33 Running Dag: dag_1463521953701_0001_1
2016-05-17 21:52:33,894 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521953701_0001_1, submitTime=1463521953856
2016-05-17 21:52:33,894 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:33,895 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F9268B2F41C84C10B3211C1CEDE04AA8
2016-05-17 21:52:33,895 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:33,896 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521953701_0001_1, initTime=1463521953895
2016-05-17 21:52:33,896 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:33,896 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953701_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:33,898 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_STARTED]: dagID=dag_1463521953701_0001_1, startTime=1463521953898
2016-05-17 21:52:33,898 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:33,898 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:33,898 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953701_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:33,899 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F9268B2F41C84C10B3211C1CEDE04AA8 : {DD2DF070A8EF4E368C0654584C76C110={InputName=DD2DF070A8EF4E368C0654584C76C110}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:33,899 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: DD2DF070A8EF4E368C0654584C76C110, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:33,899 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,899 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,899 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,899 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463521953701_0001, dagName=[5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33,899 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]. Starting root input initializers: 1
2016-05-17 21:52:33,900 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:33,900 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: DD2DF070A8EF4E368C0654584C76C110 on vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,902 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:33,903 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input DD2DF070A8EF4E368C0654584C76C110 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:33,904 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:33,904 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:33,907 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 412
2016-05-17 21:52:33,907 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: DD2DF070A8EF4E368C0654584C76C110 on vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,907 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:33,908 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {DD2DF070A8EF4E368C0654584C76C110=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:33,908 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] parallelism set to 1
2016-05-17 21:52:33,908 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:33,908 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521953701_0001_1_00, eventCount=1
2016-05-17 21:52:33,908 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,908 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1EE9BECA5D384CC9827CC8E37EFE0311, vertexId=vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:33,909 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1EE9BECA5D384CC9827CC8E37EFE0311, vertexId=vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,910 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1EE9BECA5D384CC9827CC8E37EFE0311, vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:33,911 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1EE9BECA5D384CC9827CC8E37EFE0311, vertexId=vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,912 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, vertexId=vertex_1463521953701_0001_1_00, initRequestedTime=1463521953899, initedTime=1463521953912, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:33,913 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] with distanceFromRoot: 0
2016-05-17 21:52:33,913 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:33,913 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:33,913 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F9268B2F41C84C10B3211C1CEDE04AA8
2016-05-17 21:52:33,914 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,914 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521953701_0001_1_00, startRequestedTime=1463521953913, startedTime=1463521953913
2016-05-17 21:52:33,915 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:33,915 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:33,916 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:33,916 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:TASK_STARTED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, taskId=task_1463521953701_0001_1_00_000000, scheduledTime=1463521953915, launchTime=1463521953915
2016-05-17 21:52:33,916 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953701_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:33,917 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,917 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: F9268B2F41C84C10B3211C1CEDE04AA8, VertexParallelism: 1, TaskAttemptID:attempt_1463521953701_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DD2DF070A8EF4E368C0654584C76C110, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1EE9BECA5D384CC9827CC8E37EFE0311, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:33,918 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953701_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:33,918 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:33,919 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:33,919 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:33,920 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:33,920 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521953701_0001_00_000001
2016-05-17 21:52:33,921 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:33,923 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521953701_0001_00_000001
2016-05-17 21:52:33,923 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:33,924 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521953701_0001, with dag id: dag_1463521953701_0001_1
2016-05-17 21:52:33,924 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:33,925 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:33,926 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521953701_0001_00_000001, launchTime=1463521953924
2016-05-17 21:52:33,927 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521953701_0001_00_000001
2016-05-17 21:52:33,928 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521953701_0001_1_00_000000_0] to container: [container_1463521953701_0001_00_000001]
2016-05-17 21:52:33,929 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:33,929 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521953701_0001_1_00_000000_0] started. Is using containerId: [container_1463521953701_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:33,929 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0, startTime=1463521953929, containerId=container_1463521953701_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521953701_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:33,930 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:33,930 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953701_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:33,930 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953701_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:33,932 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521953701_0001_00_000001 given task: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,932 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521953701_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33 Starting to run new task attempt: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,933 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,933 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,935 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:33,935 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:33,935 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:33,935 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:33,937 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:33,937 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: F9268B2F41C84C10B3211C1CEDE04AA8, VertexParallelism: 1, TaskAttemptID:attempt_1463521953701_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DD2DF070A8EF4E368C0654584C76C110, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1EE9BECA5D384CC9827CC8E37EFE0311, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:33,937 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:33,939 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,940 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521953701_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:33,939 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:33,940 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,941 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,941 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1EE9BECA5D384CC9827CC8E37EFE0311, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:33,941 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DD2DF070A8EF4E368C0654584C76C110, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:33,943 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:33,943 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,943 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,943 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:33,944 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1EE9BECA5D384CC9827CC8E37EFE0311
2016-05-17 21:52:33,944 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:33,944 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1EE9BECA5D384CC9827CC8E37EFE0311
2016-05-17 21:52:33,946 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:33,946 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,946 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,948 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:33,958 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9268B2F41C84C10B3211C1CEDE04AA8, ordinal: 0
2016-05-17 21:52:33,958 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9268B2F41C84C10B3211C1CEDE04AA8, mem on start (mb), free: 163, total: 555, max: 672
2016-05-17 21:52:33,958 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,958 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:33,958 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:33,959 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:33,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1EE9BECA5D384CC9827CC8E37EFE0311
2016-05-17 21:52:33,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:33,959 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:33,959 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:33,959 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:33,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:33,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:33,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:33,960 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:33,960 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DD2DF070A8EF4E368C0654584C76C110, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:33,960 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1EE9BECA5D384CC9827CC8E37EFE0311, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:33,960 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:33,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DD2DF070A8EF4E368C0654584C76C110 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:33,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:33,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:33,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,961 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:33,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,961 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:33,962 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,963 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:33,965 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 21:52:33,965 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@19d0954e
2016-05-17 21:52:33,966 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,966 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"], id: 1EE9BECA5D384CC9827CC8E37EFE0311
2016-05-17 21:52:33,966 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-17 21:52:33,967 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 21:52:33,967 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:33,967 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@19d0954e
2016-05-17 21:52:33,968 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:33,968 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:33,968 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9268B2F41C84C10B3211C1CEDE04AA8, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:33,970 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:33,970 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:33,971 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,971 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521953701_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:33,972 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,972 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9268B2F41C84C10B3211C1CEDE04AA8, mem on close (mb), free: 161, total: 555, max: 672
2016-05-17 21:52:33 Completed running task attempt: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,973 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:33,975 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0, startTime=1463521953929, finishTime=1463521953974, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 21:52:33,975 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953701_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:33,975 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:33,975 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,976 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:33,976 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:33,977 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:TASK_FINISHED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, taskId=task_1463521953701_0001_1_00_000000, startTime=1463521953929, finishTime=1463521953975, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463521953701_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 21:52:33,977 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953701_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:33,977 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] : 1
2016-05-17 21:52:33,978 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,979 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:33,980 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:VERTEX_FINISHED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, vertexId=vertex_1463521953701_0001_1_00, initRequestedTime=1463521953899, initedTime=1463521953912, startRequestedTime=1463521953913, startedTime=1463521953913, finishTime=1463521953978, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1463521953929, firstTasksToStart=[ task_1463521953701_0001_1_00_000000 ], lastTaskFinishTime=1463521953974, lastTasksToFinish=[ task_1463521953701_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521953701_0001_1_00_000000 ], longestDurationTasks=[ task_1463521953701_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:33,980 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:33,980 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:33,981 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521953701_0001_00_000001, stoppedTime=1463521953980, exitStatus=0
2016-05-17 21:52:33,981 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:33,981 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:33,981 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-05-17 21:52:33,981 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:33,981 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521953701_0001_1
2016-05-17 21:52:33,981 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521953701_0001_1
2016-05-17 21:52:33,982 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1EE9BECA5D384CC9827CC8E37EFE0311 for vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,983 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:33,984 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,984 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521953701_0001_00_000001
2016-05-17 21:52:33,985 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521953701_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:33,985 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521953701_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:33,985 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521953701_0001_00_000001, Exiting
2016-05-17 21:52:33,985 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521953701_0001_00_000001
2016-05-17 21:52:33,985 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521953701_0001_00_000001
2016-05-17 21:52:33,990 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521953701_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:33,990 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521953701_0001_00_000001 completed successfully
2016-05-17 21:52:33,995 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521953701_0001_1, queueSize=0
2016-05-17 21:52:33,995 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:33,996 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521953701_0001_1, startTime=1463521953898, finishTime=1463521953993, timeTaken=95, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 21:52:33,996 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521953701_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:33,996 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953701_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:33,997 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521953701_0001_00_000001
2016-05-17 21:52:33,997 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:33 Completed Dag: dag_1463521953701_0001_1
2016-05-17 21:52:33,999 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521953701_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:33,999 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521953701_0001_1_post
2016-05-17 21:52:34,000 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,000 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:34,004 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463521953701_0001
2016-05-17 21:52:34,006 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:34,006 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:34,006 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:34,006 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:34,006 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:34,007 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:34,007 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:34,007 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:34,007 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:34,008 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:34,008 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:34,008 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:34,008 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:34,013 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001
2016-05-17 21:52:34,013 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped/_temporary
2016-05-17 21:52:34,014 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.334
2016-05-17 21:52:34,052 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-17 21:52:34,087 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:34,088 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:34,147 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 21:52:34,166 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.048
2016-05-17 21:52:34,166 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:34,167 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:34,167 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:34,191 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-05-17 21:52:34,208 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 21:52:34,217 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-17 21:52:34,360 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-17 21:52:34,361 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:34,361 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:34,362 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-17 21:52:34,362 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  parallel execution of steps is enabled: false
2016-05-17 21:52:34,362 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  executing total steps: 1
2016-05-17 21:52:34,363 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  allocating management threads: 1
2016-05-17 21:52:34,364 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34,364 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:34,381 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-17 21:52:34,386 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:34,387 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:34,387 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:34,396 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001 doesn't exist and is created
2016-05-17 21:52:34,415 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:34,416 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:34,416 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001
2016-05-17 21:52:34,421 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521954386_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:34,436 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:34,437 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:34,439 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:34,439 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:34,439 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001/localmode-log-dir
2016-05-17 21:52:34,439 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001/localmode-log-dir/history.txt.appattempt_1463521954386_0001_000000, maxErrors=10
2016-05-17 21:52:34,439 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:34,440 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521954386_0001_000000, appSubmitTime=1463521954420, launchTime=1463521954420
2016-05-17 21:52:34,447 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:34,448 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:34,450 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38515
2016-05-17 21:52:34,451 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:34,451 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:34,452 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:34,454 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:34,455 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521954386_0001_000000, startTime=1463521954454
2016-05-17 21:52:34,455 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:34,516 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:34,516 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:34,517 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463521954386_0001, dagName=[1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34,543 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34,545 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521954386_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001/localmode-log-dir/dag_1463521954386_0001_1.dot
2016-05-17 21:52:34,549 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001/localmode-log-dir/dag_1463521954386_0001_1-tez-dag.pb.txt
2016-05-17 21:52:34,581 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521954386_0001_1
2016-05-17 21:52:34,581 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,582 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34 Running Dag: dag_1463521954386_0001_1
2016-05-17 21:52:34,603 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521954386_0001_1, submitTime=1463521954543
2016-05-17 21:52:34,603 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:34,604 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,605 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:34,605 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521954386_0001_1, initTime=1463521954603
2016-05-17 21:52:34,607 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521954386_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:34,605 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:34,608 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_STARTED]: dagID=dag_1463521954386_0001_1, startTime=1463521954608
2016-05-17 21:52:34,608 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:34,609 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521954386_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:34,609 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C470363514604326B704E08A2AE6FAD0 : {9191FB124AAA460AB8A9F7EBBC3E20E9={InputName=9191FB124AAA460AB8A9F7EBBC3E20E9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:34,609 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9191FB124AAA460AB8A9F7EBBC3E20E9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:34,609 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,609 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,610 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,610 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]. Starting root input initializers: 1
2016-05-17 21:52:34,608 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:34,614 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:34,615 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9191FB124AAA460AB8A9F7EBBC3E20E9 on vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,615 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D00E6B9D4D534639BBBB9445F8224123 : {776565AC6F3E4415AD6BDC4293BCBA86={InputName=776565AC6F3E4415AD6BDC4293BCBA86}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:34,615 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 776565AC6F3E4415AD6BDC4293BCBA86, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:34,616 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,616 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,616 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,616 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]. Starting root input initializers: 1
2016-05-17 21:52:34,617 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463521954386_0001, dagName=[1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34,617 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:34,618 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,618 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 776565AC6F3E4415AD6BDC4293BCBA86 on vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,622 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:34,622 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9191FB124AAA460AB8A9F7EBBC3E20E9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:34,623 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:34,623 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:34,626 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,626 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,627 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,627 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CA127B987753482BAD9D4BC705A52257, vertexId=vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:34,627 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CA127B987753482BAD9D4BC705A52257, vertexId=vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,626 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 776565AC6F3E4415AD6BDC4293BCBA86 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:34,629 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:34,630 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:34,626 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:34,633 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:34,633 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CA127B987753482BAD9D4BC705A52257, vertexName=B84D2ABBACF84A739BC24F619B6792E9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:34,634 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CA127B987753482BAD9D4BC705A52257, vertexId=vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,635 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:34,639 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9191FB124AAA460AB8A9F7EBBC3E20E9 on vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,640 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:34,640 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 776565AC6F3E4415AD6BDC4293BCBA86 on vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,642 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:34,642 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, vertexId=vertex_1463521954386_0001_1_02, initRequestedTime=1463521954618, initedTime=1463521954641, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:34,643 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:34,643 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:34,645 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9191FB124AAA460AB8A9F7EBBC3E20E9=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:34,645 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] parallelism set to 1
2016-05-17 21:52:34,645 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:34,647 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521954386_0001_1_01, eventCount=1
2016-05-17 21:52:34,649 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:34,649 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C470363514604326B704E08A2AE6FAD0, vertexId=vertex_1463521954386_0001_1_01, initRequestedTime=1463521954609, initedTime=1463521954648, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:34,649 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] with distanceFromRoot: 0
2016-05-17 21:52:34,650 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:34,650 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:34,650 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {776565AC6F3E4415AD6BDC4293BCBA86=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:34,650 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] parallelism set to 1
2016-05-17 21:52:34,651 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:34,654 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521954386_0001_1_00, eventCount=1
2016-05-17 21:52:34,655 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:34,655 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, vertexId=vertex_1463521954386_0001_1_00, initRequestedTime=1463521954615, initedTime=1463521954654, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:34,655 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] with distanceFromRoot: 0
2016-05-17 21:52:34,655 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:34,655 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,655 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,657 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463521954386_0001, with dag id: dag_1463521954386_0001_1
2016-05-17 21:52:34,657 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:34,657 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521954386_0001_1_01, startRequestedTime=1463521954655, startedTime=1463521954655
2016-05-17 21:52:34,660 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:34,661 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,661 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,661 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:34,661 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521954386_0001_1_00, startRequestedTime=1463521954661, startedTime=1463521954661
2016-05-17 21:52:34,662 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:34,664 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:34,664 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_STARTED]: vertexName=C470363514604326B704E08A2AE6FAD0, taskId=task_1463521954386_0001_1_01_000000, scheduledTime=1463521954664, launchTime=1463521954664
2016-05-17 21:52:34,665 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:34,665 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521954386_0001_1_01 for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] numStartedSources: 1 numSources: 2
2016-05-17 21:52:34,665 INFO  impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] numStartedSources: 1 numSources: 2
2016-05-17 21:52:34,669 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:34,669 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_STARTED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, taskId=task_1463521954386_0001_1_00_000000, scheduledTime=1463521954669, launchTime=1463521954669
2016-05-17 21:52:34,670 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:34,671 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521954386_0001_1_00 for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] numStartedSources: 2 numSources: 2
2016-05-17 21:52:34,672 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] with distanceFromRoot: 1
2016-05-17 21:52:34,673 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,673 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,673 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D00E6B9D4D534639BBBB9445F8224123 in vertex: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,673 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C470363514604326B704E08A2AE6FAD0 in vertex: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,673 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B84D2ABBACF84A739BC24F619B6792E9 with 2 source tasks and 1 pending tasks
2016-05-17 21:52:34,674 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:34,674 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521954386_0001_1_02, startRequestedTime=1463521954673, startedTime=1463521954673
2016-05-17 21:52:34,674 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:34,674 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: C470363514604326B704E08A2AE6FAD0, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9191FB124AAA460AB8A9F7EBBC3E20E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:34,675 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:34,675 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: D00E6B9D4D534639BBBB9445F8224123, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=776565AC6F3E4415AD6BDC4293BCBA86, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:34,675 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:34,675 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:34,676 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:34,676 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:34,677 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:34,677 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:34,677 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521954386_0001_00_000001
2016-05-17 21:52:34,678 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:34,679 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521954386_0001_00_000002
2016-05-17 21:52:34,679 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:34,681 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521954386_0001_00_000001
2016-05-17 21:52:34,681 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:34,682 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:34,682 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:34,683 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521954386_0001_00_000001, launchTime=1463521954682
2016-05-17 21:52:34,684 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521954386_0001_00_000002
2016-05-17 21:52:34,687 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000001
2016-05-17 21:52:34,687 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:34,689 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521954386_0001_1_01_000000_0] to container: [container_1463521954386_0001_00_000001]
2016-05-17 21:52:34,690 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:34,691 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521954386_0001_1_01_000000_0] started. Is using containerId: [container_1463521954386_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:34,691 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C470363514604326B704E08A2AE6FAD0, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0, startTime=1463521954690, containerId=container_1463521954386_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521954386_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:34,692 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:34,692 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:34,692 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:34,693 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:34,693 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521954386_0001_00_000002, launchTime=1463521954692
2016-05-17 21:52:34,693 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:34,694 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521954386_0001_00_000001 given task: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,694 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,697 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000002
2016-05-17 21:52:34 Starting to run new task attempt: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,697 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,698 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,698 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:34,699 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:34,699 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:34,699 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:34,699 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:34,699 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: C470363514604326B704E08A2AE6FAD0, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9191FB124AAA460AB8A9F7EBBC3E20E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:34,699 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:34,704 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,704 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:34,704 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,705 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521954386_0001_1_00_000000_0] to container: [container_1463521954386_0001_00_000002]
2016-05-17 21:52:34,706 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:34,707 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521954386_0001_1_00_000000_0] started. Is using containerId: [container_1463521954386_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:34,707 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0, startTime=1463521954706, containerId=container_1463521954386_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521954386_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:34,707 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:34,707 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:34,707 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:34,708 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521954386_0001_00_000002 given task: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,708 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,709 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,742 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9191FB124AAA460AB8A9F7EBBC3E20E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:34,743 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:34,743 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,743 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,745 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521954386_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:34 Starting to run new task attempt: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,748 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,748 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,749 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:34,750 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:34,750 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:34,750 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:34,750 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:34,750 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: D00E6B9D4D534639BBBB9445F8224123, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=776565AC6F3E4415AD6BDC4293BCBA86, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:34,750 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:34,751 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:34,751 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,751 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:34,751 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,751 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:34,752 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,753 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:34,753 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=776565AC6F3E4415AD6BDC4293BCBA86, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:34,754 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:34,754 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,754 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:34,756 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,756 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,757 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,757 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:34,762 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:34,764 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:34,764 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,766 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:34,766 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,768 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:34,768 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,768 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,767 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,770 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:34,768 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521954386_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:34,772 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,773 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,772 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C470363514604326B704E08A2AE6FAD0, ordinal: 0
2016-05-17 21:52:34,774 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C470363514604326B704E08A2AE6FAD0, mem on start (mb), free: 256, total: 554, max: 672
2016-05-17 21:52:34,774 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,775 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:34,775 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:34,776 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:34,776 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:34,777 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:34,778 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:34,778 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:34,778 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:34,779 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:34,780 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9191FB124AAA460AB8A9F7EBBC3E20E9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:34,784 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B84D2ABBACF84A739BC24F619B6792E9, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:34,784 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:34,785 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9191FB124AAA460AB8A9F7EBBC3E20E9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:34,785 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:34,786 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:34,771 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,785 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,787 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:34,787 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,787 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:34,776 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D00E6B9D4D534639BBBB9445F8224123, ordinal: 1
2016-05-17 21:52:34,788 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D00E6B9D4D534639BBBB9445F8224123, mem on start (mb), free: 256, total: 554, max: 672
2016-05-17 21:52:34,788 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,788 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:34,788 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,788 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:34,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:34,789 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:34,789 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:34,789 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:34,789 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:34,790 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:34,790 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:34,790 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 776565AC6F3E4415AD6BDC4293BCBA86, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:34,790 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B84D2ABBACF84A739BC24F619B6792E9, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:34,790 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:34,790 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 776565AC6F3E4415AD6BDC4293BCBA86 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:34,790 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:34,791 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:34,788 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:34,792 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,792 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:34,793 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:34,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,796 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,796 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:34,796 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:34,798 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:34,800 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@66d182d7
2016-05-17 21:52:34,801 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,804 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:34,804 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1462f077; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6b991f15; comparator=cascading.tuple.hadoop.util.TupleComparator@48d8fd10; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:34,804 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,804 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@66d182d7
2016-05-17 21:52:34,806 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:34,806 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:34,806 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:34,807 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:34,807 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:34,805 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:34,808 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:34,818 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2831bb22; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@58f4c9cb; comparator=cascading.tuple.hadoop.util.TupleComparator@2eec5c0d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:34,806 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:34,825 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:34,828 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:34,828 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:34,829 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:34,829 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:34,825 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:34,829 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:34,829 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:34,828 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@63ce2e87
2016-05-17 21:52:34,834 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:34,834 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:34,835 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@63ce2e87
2016-05-17 21:52:34,835 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:34,837 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:34,835 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C470363514604326B704E08A2AE6FAD0, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:34,854 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:34,855 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:34,856 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:34,856 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:34,856 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:34,857 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:34,857 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:34,858 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D00E6B9D4D534639BBBB9445F8224123, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:34,860 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,861 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,862 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D00E6B9D4D534639BBBB9445F8224123, mem on close (mb), free: 53, total: 554, max: 672
2016-05-17 21:52:34,862 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:34,862 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:34,862 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:34,863 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:34,861 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C470363514604326B704E08A2AE6FAD0, mem on close (mb), free: 53, total: 554, max: 672
2016-05-17 21:52:34,865 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:34,865 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:34,865 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:34,867 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:34,869 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:34,872 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:34 Completed running task attempt: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,875 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:34,875 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521954386_0001_1_00, eventCount=1
2016-05-17 21:52:34,876 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0, startTime=1463521954706, finishTime=1463521954876, timeTaken=170, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=222, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954858, Process_Duration=3, Process_End_Time=1463521954861, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:34,877 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:34,877 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:34,877 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:34,877 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,881 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:34,881 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_FINISHED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, taskId=task_1463521954386_0001_1_00_000000, startTime=1463521954706, finishTime=1463521954880, timeTaken=174, status=SUCCEEDED, successfulAttemptID=attempt_1463521954386_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=222, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954858, Process_Duration=3, Process_End_Time=1463521954861, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:34,881 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:34,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=222, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521954858, Process_Duration=3, Process_End_Time=1463521954861, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:34,882 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] : 1
2016-05-17 21:52:34,883 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,885 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:34,885 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_FINISHED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, vertexId=vertex_1463521954386_0001_1_00, initRequestedTime=1463521954615, initedTime=1463521954654, startRequestedTime=1463521954661, startedTime=1463521954661, finishTime=1463521954883, timeTaken=222, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=222, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954858, Process_Duration=3, Process_End_Time=1463521954861, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521954706, firstTasksToStart=[ task_1463521954386_0001_1_00_000000 ], lastTaskFinishTime=1463521954876, lastTasksToFinish=[ task_1463521954386_0001_1_00_000000 ], minTaskDuration=170, maxTaskDuration=170, avgTaskDuration=170.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521954386_0001_1_00_000000 ], longestDurationTasks=[ task_1463521954386_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:34,885 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:34,886 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:34,887 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:34,887 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521954386_0001_00_000002, stoppedTime=1463521954886, exitStatus=0
2016-05-17 21:52:34,887 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:34,887 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] attempt: attempt_1463521954386_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:34,888 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:34,888 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521954386_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:52:34,889 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:34,889 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:34,889 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,890 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000002
2016-05-17 21:52:34,891 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521954386_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:34,891 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:34,891 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521954386_0001_00_000002, Exiting
2016-05-17 21:52:34,891 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521954386_0001_00_000002
2016-05-17 21:52:34,891 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521954386_0001_00_000002
2016-05-17 21:52:34,893 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521954386_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:34,894 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521954386_0001_00_000002 completed successfully
2016-05-17 21:52:34 Completed running task attempt: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,895 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:34,897 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:34,898 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521954386_0001_1_01, eventCount=1
2016-05-17 21:52:34,899 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:34,899 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C470363514604326B704E08A2AE6FAD0, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0, startTime=1463521954690, finishTime=1463521954899, timeTaken=209, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=31, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954838, Process_Duration=22, Process_End_Time=1463521954860, Read_Duration=20, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:34,900 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=31, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521954838, Process_Duration=22, Process_End_Time=1463521954860, Read_Duration=20, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:34,900 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:34,900 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,901 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000001
2016-05-17 21:52:34,903 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:34,903 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:34,903 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,904 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:34,905 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_FINISHED]: vertexName=C470363514604326B704E08A2AE6FAD0, taskId=task_1463521954386_0001_1_01_000000, startTime=1463521954690, finishTime=1463521954904, timeTaken=214, status=SUCCEEDED, successfulAttemptID=attempt_1463521954386_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=31, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954838, Process_Duration=22, Process_End_Time=1463521954860, Read_Duration=20, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:34,905 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:34,905 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] : 1
2016-05-17 21:52:34,906 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,908 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:34,909 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_FINISHED]: vertexName=C470363514604326B704E08A2AE6FAD0, vertexId=vertex_1463521954386_0001_1_01, initRequestedTime=1463521954609, initedTime=1463521954648, startRequestedTime=1463521954655, startedTime=1463521954655, finishTime=1463521954906, timeTaken=251, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=31, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954838, Process_Duration=22, Process_End_Time=1463521954860, Read_Duration=20, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521954690, firstTasksToStart=[ task_1463521954386_0001_1_01_000000 ], lastTaskFinishTime=1463521954899, lastTasksToFinish=[ task_1463521954386_0001_1_01_000000 ], minTaskDuration=209, maxTaskDuration=209, avgTaskDuration=209.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521954386_0001_1_01_000000 ], longestDurationTasks=[ task_1463521954386_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:34,910 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:34,911 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:34,912 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:34,912 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521954386_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:34,913 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:34,913 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521954386_0001_00_000001, Exiting
2016-05-17 21:52:34,913 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521954386_0001_00_000001
2016-05-17 21:52:34,913 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521954386_0001_00_000001
2016-05-17 21:52:34,914 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521954386_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:34,914 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521954386_0001_00_000001 completed successfully
2016-05-17 21:52:34,912 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521954386_0001_00_000001, stoppedTime=1463521954911, exitStatus=0
2016-05-17 21:52:34,915 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:34,915 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] attempt: attempt_1463521954386_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:34,916 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,916 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,917 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:34,917 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:34,918 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521954386_0001_00_000001
2016-05-17 21:52:34,919 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_STARTED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, taskId=task_1463521954386_0001_1_02_000000, scheduledTime=1463521954919, launchTime=1463521954919
2016-05-17 21:52:34,919 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:34,919 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:34,920 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,921 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: B84D2ABBACF84A739BC24F619B6792E9, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D00E6B9D4D534639BBBB9445F8224123, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C470363514604326B704E08A2AE6FAD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CA127B987753482BAD9D4BC705A52257, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:34,921 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:34,922 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:34,923 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521954386_0001_00_000003
2016-05-17 21:52:34,924 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:34,927 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521954386_0001_00_000003
2016-05-17 21:52:34,928 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:34,931 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:34,931 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:34,932 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521954386_0001_00_000003, launchTime=1463521954931
2016-05-17 21:52:34,933 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000003
2016-05-17 21:52:34,934 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521954386_0001_1_02_000000_0] to container: [container_1463521954386_0001_00_000003]
2016-05-17 21:52:34,934 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:34,935 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521954386_0001_1_02_000000_0] started. Is using containerId: [container_1463521954386_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 21:52:34,935 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0, startTime=1463521954935, containerId=container_1463521954386_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521954386_0001_00_000003/teamcity, completedLogs=
2016-05-17 21:52:34,936 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:34,936 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:34,936 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:34,937 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521954386_0001_00_000003 given task: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34,937 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34 Starting to run new task attempt: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34,939 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34,939 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,939 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:34,939 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:34,939 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:34,940 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:34,940 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:34,940 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: B84D2ABBACF84A739BC24F619B6792E9, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D00E6B9D4D534639BBBB9445F8224123, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C470363514604326B704E08A2AE6FAD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CA127B987753482BAD9D4BC705A52257, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:34,940 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:34,940 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34,941 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:34,941 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,943 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D00E6B9D4D534639BBBB9445F8224123, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:34,944 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:34,944 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,946 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:34,947 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:34,947 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,947 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,949 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521954386_0001_1_02_000000_0 sent events: (0-2)
2016-05-17 21:52:34,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C470363514604326B704E08A2AE6FAD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:34,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:34,951 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,952 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:34,953 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:34,953 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,954 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,954 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CA127B987753482BAD9D4BC705A52257, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:34,956 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:34,956 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:34,956 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CA127B987753482BAD9D4BC705A52257
2016-05-17 21:52:34,957 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:34,957 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CA127B987753482BAD9D4BC705A52257
2016-05-17 21:52:34,964 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B84D2ABBACF84A739BC24F619B6792E9, ordinal: 2
2016-05-17 21:52:34,964 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B84D2ABBACF84A739BC24F619B6792E9, mem on start (mb), free: 44, total: 554, max: 672
2016-05-17 21:52:34,964 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,964 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-05-17 21:52:34,964 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:34,964 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:34,991 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:34,991 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CA127B987753482BAD9D4BC705A52257
2016-05-17 21:52:34,991 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:34,992 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:34,992 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:34,992 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:34,992 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:34,992 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:34,992 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-17 21:52:34,993 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-05-17 21:52:34,993 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D00E6B9D4D534639BBBB9445F8224123, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 21:52:34,993 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C470363514604326B704E08A2AE6FAD0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 21:52:34,993 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CA127B987753482BAD9D4BC705A52257, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:34,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:34,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D00E6B9D4D534639BBBB9445F8224123 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:34,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C470363514604326B704E08A2AE6FAD0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:34,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-05-17 21:52:34,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-05-17 21:52:34,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,994 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:34,994 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:34,995 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:34,995 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D00E6B9D4D534639BBBB9445F8224123 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:34,995 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:34,995 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 21:52:34,995 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:34,995 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C470363514604326B704E08A2AE6FAD0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:34,995 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:34,995 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 21:52:34,996 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:34,998 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,999 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:34,999 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,999 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:35,000 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:35,000 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,004 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521954386_0001_1_00_000000_0_10027, runDuration: 107533, ]
2016-05-17 21:52:35,005 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521954386_0001_1_01_000000_0_10024, runDuration: 137200, ]
2016-05-17 21:52:35,006 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]]
2016-05-17 21:52:35,007 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] streamed: true, id: 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:35,007 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"], id: CA127B987753482BAD9D4BC705A52257
2016-05-17 21:52:35,008 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-17 21:52:35,008 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 866FFC915F87476284463FCA2CD8A30D, for 2 inputs
2016-05-17 21:52:35,009 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 866FFC915F87476284463FCA2CD8A30D, for 2 inputs
2016-05-17 21:52:35,010 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521954386_0001_1_01_000000_0_10024] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521955010, TimeTaken=3, Rate=0.03 MB/s
2016-05-17 21:52:35,010 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:35,011 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:35,011 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C470363514604326B704E08A2AE6FAD0] #4 in 4ms
2016-05-17 21:52:35,017 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C470363514604326B704E08A2AE6FAD0] #4, status:false, isInterrupted:false
2016-05-17 21:52:35,018 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521954386_0001_1_00_000000_0_10027] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521955018, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 21:52:35,018 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:35,019 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:35,019 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D00E6B9D4D534639BBBB9445F8224123] #5 in 2ms
2016-05-17 21:52:35,019 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D00E6B9D4D534639BBBB9445F8224123] #5, status:false, isInterrupted:false
2016-05-17 21:52:35,021 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:35,023 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:35,023 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,023 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:35,023 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:35,023 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:35,024 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:35,025 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:35,025 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,025 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:35,026 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:35,026 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:35,030 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 21:52:35,030 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 21:52:35,032 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:35,033 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:35,036 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B84D2ABBACF84A739BC24F619B6792E9, all 2 inputs ready in: 00:00:00.000
2016-05-17 21:52:35,040 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5813c035; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@78b004bf
2016-05-17 21:52:35,043 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@6c07d57f; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7651bc41
2016-05-17 21:52:35,048 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:35,049 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:35,050 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,051 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521954386_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 21:52:35,052 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,052 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B84D2ABBACF84A739BC24F619B6792E9, mem on close (mb), free: 259, total: 555, max: 672
2016-05-17 21:52:35,053 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:35,053 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:35,054 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:35,055 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,055 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:35,055 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:35,056 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:35,056 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:35,058 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:35,058 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,058 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:35,059 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:35 Completed running task attempt: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,060 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:35,062 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0, startTime=1463521954935, finishTime=1463521955062, timeTaken=127, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955037, Process_Duration=15, Process_End_Time=1463521955052, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:35,062 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:35,063 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:35,063 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,064 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_FINISHED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, taskId=task_1463521954386_0001_1_02_000000, startTime=1463521954935, finishTime=1463521955063, timeTaken=128, status=SUCCEEDED, successfulAttemptID=attempt_1463521954386_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955037, Process_Duration=15, Process_End_Time=1463521955052, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:35,064 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:35,064 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] : 1
2016-05-17 21:52:35,064 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:35,066 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:35,067 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:35,069 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_FINISHED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, vertexId=vertex_1463521954386_0001_1_02, initRequestedTime=1463521954618, initedTime=1463521954641, startRequestedTime=1463521954673, startedTime=1463521954673, finishTime=1463521955064, timeTaken=391, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955037, Process_Duration=15, Process_End_Time=1463521955052, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463521954935, firstTasksToStart=[ task_1463521954386_0001_1_02_000000 ], lastTaskFinishTime=1463521955062, lastTasksToFinish=[ task_1463521954386_0001_1_02_000000 ], minTaskDuration=127, maxTaskDuration=127, avgTaskDuration=127.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521954386_0001_1_02_000000 ], longestDurationTasks=[ task_1463521954386_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:35,070 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:35,070 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:35,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521955037, Process_Duration=15, Process_End_Time=1463521955052, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 21:52:35,073 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:35,073 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,075 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000003
2016-05-17 21:52:35,078 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:35,078 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521954386_0001_00_000003, stoppedTime=1463521955078, exitStatus=0
2016-05-17 21:52:35,078 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:35,078 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:35,081 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:35,081 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521954386_0001_1
2016-05-17 21:52:35,081 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521954386_0001_1
2016-05-17 21:52:35,081 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CA127B987753482BAD9D4BC705A52257 for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:35,088 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521954386_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 21:52:35,088 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:35,089 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521954386_0001_00_000003, Exiting
2016-05-17 21:52:35,089 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521954386_0001_00_000003
2016-05-17 21:52:35,089 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521954386_0001_00_000003
2016-05-17 21:52:35,090 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521954386_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:35,090 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521954386_0001_00_000003 completed successfully
2016-05-17 21:52:35,093 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:35,093 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:35,096 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521954386_0001_1, queueSize=0
2016-05-17 21:52:35,096 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:35,097 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521954386_0001_1, startTime=1463521954608, finishTime=1463521955093, timeTaken=485, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=470, FILE_BYTES_WRITTEN=8276, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=51, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1743781888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390565864733, Process_Duration=40, Process_End_Time=4390565864773, Read_Duration=23, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-17 21:52:35,097 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521954386_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:35,097 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521954386_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:35,100 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521954386_0001_00_000003
2016-05-17 21:52:35,100 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:35 Completed Dag: dag_1463521954386_0001_1
2016-05-17 21:52:35,101 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521954386_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:35,102 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521954386_0001_1_post
2016-05-17 21:52:35,102 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,102 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:35,107 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463521954386_0001
2016-05-17 21:52:35,110 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:35,110 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:35,110 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:35,110 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:35,110 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:35,113 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:35,113 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:35,116 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:35,116 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:35,116 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:35,117 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:35,118 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:35,119 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:35,126 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001
2016-05-17 21:52:35,127 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown/_temporary
2016-05-17 21:52:35,127 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  completed in: 00:00.766
2016-05-17 21:52:35,211 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-17 21:52:35,237 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:35,237 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:35,290 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 21:52:35,303 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-05-17 21:52:35,304 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:35,304 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:35,305 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:35,325 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:35,335 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-17 21:52:35,386 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:35,387 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:35,387 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-17 21:52:35,387 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:35,387 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:35,388 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:35,388 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...leRelative/simplerelative
2016-05-17 21:52:35,389 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:35,401 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:35,407 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:35,408 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:35,408 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:35,417 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001 doesn't exist and is created
2016-05-17 21:52:35,434 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:35,435 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:35,435 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001
2016-05-17 21:52:35,438 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521955407_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:35,450 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:35,451 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:35,452 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:35,452 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:35,453 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001/localmode-log-dir
2016-05-17 21:52:35,453 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001/localmode-log-dir/history.txt.appattempt_1463521955407_0001_000000, maxErrors=10
2016-05-17 21:52:35,453 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:35,453 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521955407_0001_000000, appSubmitTime=1463521955438, launchTime=1463521955438
2016-05-17 21:52:35,458 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:35,461 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:35,464 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:35,464 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48986
2016-05-17 21:52:35,466 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:35,467 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:35,469 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521955407_0001_000000, startTime=1463521955469
2016-05-17 21:52:35,469 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:35,469 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:35,535 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:35,535 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:35,535 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463521955407_0001, dagName=[3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative
2016-05-17 21:52:35,556 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative
2016-05-17 21:52:35,558 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521955407_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001/localmode-log-dir/dag_1463521955407_0001_1.dot
2016-05-17 21:52:35,559 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001/localmode-log-dir/dag_1463521955407_0001_1-tez-dag.pb.txt
2016-05-17 21:52:35,571 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521955407_0001_1
2016-05-17 21:52:35,571 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,572 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative
2016-05-17 21:52:35 Running Dag: dag_1463521955407_0001_1
2016-05-17 21:52:35,585 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521955407_0001_1, submitTime=1463521955556
2016-05-17 21:52:35,585 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:35,586 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,586 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:35,587 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:35,587 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521955407_0001_1, initTime=1463521955585
2016-05-17 21:52:35,587 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521955407_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:35,588 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_STARTED]: dagID=dag_1463521955407_0001_1, startTime=1463521955588
2016-05-17 21:52:35,588 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:35,588 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:35,588 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521955407_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:35,589 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E1F5DCBD4B27474A9A5DFE6A8726B3EB : {B2EAB128DFDC4181BFA9D885CEB348C3={InputName=B2EAB128DFDC4181BFA9D885CEB348C3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:35,589 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B2EAB128DFDC4181BFA9D885CEB348C3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:35,589 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,589 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,590 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,590 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]. Starting root input initializers: 1
2016-05-17 21:52:35,590 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463521955407_0001, dagName=[3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative
2016-05-17 21:52:35,590 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:35,591 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,591 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B2EAB128DFDC4181BFA9D885CEB348C3 on vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,593 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:35,593 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B2EAB128DFDC4181BFA9D885CEB348C3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:35,594 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:35,594 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:35,595 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:35,595 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,595 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,595 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,595 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A1403853B0D3429EAC6D6BD22BC4D2D8, vertexId=vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:35,596 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A1403853B0D3429EAC6D6BD22BC4D2D8, vertexId=vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,597 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:35,598 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B2EAB128DFDC4181BFA9D885CEB348C3 on vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,600 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A1403853B0D3429EAC6D6BD22BC4D2D8, vertexName=F6AB05418EB14BBBA8BA4F876059A456, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:35,601 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A1403853B0D3429EAC6D6BD22BC4D2D8, vertexId=vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,603 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:35,603 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, vertexId=vertex_1463521955407_0001_1_01, initRequestedTime=1463521955591, initedTime=1463521955603, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:35,603 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:35,603 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:35,604 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B2EAB128DFDC4181BFA9D885CEB348C3=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:35,604 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] parallelism set to 1
2016-05-17 21:52:35,604 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:35,605 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521955407_0001_1_00, eventCount=1
2016-05-17 21:52:35,606 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:35,606 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, vertexId=vertex_1463521955407_0001_1_00, initRequestedTime=1463521955589, initedTime=1463521955605, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:35,606 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] with distanceFromRoot: 0
2016-05-17 21:52:35,607 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:35,607 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,607 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,608 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521955407_0001_1_00, startRequestedTime=1463521955607, startedTime=1463521955607
2016-05-17 21:52:35,608 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:35,608 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:35,608 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:35,608 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_STARTED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, taskId=task_1463521955407_0001_1_00_000000, scheduledTime=1463521955608, launchTime=1463521955608
2016-05-17 21:52:35,609 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:35,609 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521955407_0001_1_00 for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] numStartedSources: 1 numSources: 1
2016-05-17 21:52:35,609 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] with distanceFromRoot: 1
2016-05-17 21:52:35,609 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,609 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E1F5DCBD4B27474A9A5DFE6A8726B3EB in vertex: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,610 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F6AB05418EB14BBBA8BA4F876059A456 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:35,610 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521955407_0001_1_01, startRequestedTime=1463521955609, startedTime=1463521955609
2016-05-17 21:52:35,610 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:35,611 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:35,611 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative, VertexName: E1F5DCBD4B27474A9A5DFE6A8726B3EB, VertexParallelism: 1, TaskAttemptID:attempt_1463521955407_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B2EAB128DFDC4181BFA9D885CEB348C3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F6AB05418EB14BBBA8BA4F876059A456, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:35,612 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:35,612 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:35,613 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:35,613 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521955407_0001, with dag id: dag_1463521955407_0001_1
2016-05-17 21:52:35,613 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:35,613 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:35,614 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521955407_0001_00_000001
2016-05-17 21:52:35,614 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:35,616 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521955407_0001_00_000001
2016-05-17 21:52:35,616 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:35,617 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:35,617 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:35,618 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521955407_0001_00_000001, launchTime=1463521955617
2016-05-17 21:52:35,620 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521955407_0001_00_000001
2016-05-17 21:52:35,622 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521955407_0001_1_00_000000_0] to container: [container_1463521955407_0001_00_000001]
2016-05-17 21:52:35,622 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:35,623 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521955407_0001_1_00_000000_0] started. Is using containerId: [container_1463521955407_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:35,624 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:35,624 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0, startTime=1463521955623, containerId=container_1463521955407_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521955407_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:35,624 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:35,624 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:35,627 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521955407_0001_00_000001 given task: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,627 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521955407_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35 Starting to run new task attempt: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,629 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,630 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,630 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:35,631 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:35,631 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:35,632 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:35,632 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:35,633 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative, VertexName: E1F5DCBD4B27474A9A5DFE6A8726B3EB, VertexParallelism: 1, TaskAttemptID:attempt_1463521955407_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B2EAB128DFDC4181BFA9D885CEB348C3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F6AB05418EB14BBBA8BA4F876059A456, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:35,633 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:35,634 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,634 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:35,634 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,636 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B2EAB128DFDC4181BFA9D885CEB348C3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:35,637 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,638 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:35,640 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,641 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,641 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521955407_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:35,644 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:35,644 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F6AB05418EB14BBBA8BA4F876059A456, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:35,644 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:35,645 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:35,645 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,647 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:35,647 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,648 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,654 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E1F5DCBD4B27474A9A5DFE6A8726B3EB, ordinal: 0
2016-05-17 21:52:35,654 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E1F5DCBD4B27474A9A5DFE6A8726B3EB, mem on start (mb), free: 163, total: 555, max: 672
2016-05-17 21:52:35,654 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,654 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:35,654 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:35,655 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:35,655 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:35,655 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:35,655 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:35,655 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:35,655 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:35,656 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:35,656 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B2EAB128DFDC4181BFA9D885CEB348C3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:35,656 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F6AB05418EB14BBBA8BA4F876059A456, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:35,656 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:35,656 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B2EAB128DFDC4181BFA9D885CEB348C3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:35,656 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:35,656 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:35,656 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,657 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:35,657 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,657 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:35,658 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,660 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:35,661 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@482ac91d
2016-05-17 21:52:35,662 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:35,663 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,663 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 22DB2FDF46894B7098BD96D37589992D
2016-05-17 21:52:35,663 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@482ac91d
2016-05-17 21:52:35,664 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:35,664 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:35,664 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 22DB2FDF46894B7098BD96D37589992D
2016-05-17 21:52:35,667 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6dfadcb; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@470aedf3; comparator=cascading.tuple.hadoop.util.TupleComparator@1e8c10f5; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:35,667 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:35,668 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:35,668 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:35,682 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:35,683 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:35,683 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:35,683 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:35,683 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:35,684 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:35,684 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:35,685 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E1F5DCBD4B27474A9A5DFE6A8726B3EB, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:35,687 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,688 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E1F5DCBD4B27474A9A5DFE6A8726B3EB, mem on close (mb), free: 61, total: 555, max: 672
2016-05-17 21:52:35,688 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:35,688 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:35,688 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 21:52:35,689 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 21:52:35,695 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:35 Completed running task attempt: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,700 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:35,701 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521955407_0001_1_00, eventCount=1
2016-05-17 21:52:35,701 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521955685, Process_Duration=2, Process_End_Time=1463521955687, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 21:52:35,702 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:35,702 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0, startTime=1463521955623, finishTime=1463521955701, timeTaken=78, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955685, Process_Duration=2, Process_End_Time=1463521955687, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:35,702 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:35,702 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:35,702 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,703 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:35,704 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:35,704 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_FINISHED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, taskId=task_1463521955407_0001_1_00_000000, startTime=1463521955623, finishTime=1463521955703, timeTaken=80, status=SUCCEEDED, successfulAttemptID=attempt_1463521955407_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955685, Process_Duration=2, Process_End_Time=1463521955687, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:35,704 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:35,705 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] : 1
2016-05-17 21:52:35,705 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,706 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:35,707 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_FINISHED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, vertexId=vertex_1463521955407_0001_1_00, initRequestedTime=1463521955589, initedTime=1463521955605, startRequestedTime=1463521955607, startedTime=1463521955607, finishTime=1463521955705, timeTaken=98, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955685, Process_Duration=2, Process_End_Time=1463521955687, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463521955623, firstTasksToStart=[ task_1463521955407_0001_1_00_000000 ], lastTaskFinishTime=1463521955701, lastTasksToFinish=[ task_1463521955407_0001_1_00_000000 ], minTaskDuration=78, maxTaskDuration=78, avgTaskDuration=78.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521955407_0001_1_00_000000 ], longestDurationTasks=[ task_1463521955407_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:35,707 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:35,707 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:35,708 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521955407_0001_00_000001, stoppedTime=1463521955707, exitStatus=0
2016-05-17 21:52:35,708 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521955407_0001_00_000001
2016-05-17 21:52:35,708 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:35,708 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] attempt: attempt_1463521955407_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:35,708 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:35,709 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,709 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,708 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521955407_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:35,710 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521955407_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:35,710 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:35,710 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521955407_0001_00_000001, Exiting
2016-05-17 21:52:35,710 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521955407_0001_00_000001
2016-05-17 21:52:35,710 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521955407_0001_00_000001
2016-05-17 21:52:35,711 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521955407_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:35,712 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_STARTED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, taskId=task_1463521955407_0001_1_01_000000, scheduledTime=1463521955711, launchTime=1463521955711
2016-05-17 21:52:35,711 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:35,712 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:35,713 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,713 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521955407_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:35,713 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:35,713 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521955407_0001_00_000001 completed successfully
2016-05-17 21:52:35,713 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative, VertexName: F6AB05418EB14BBBA8BA4F876059A456, VertexParallelism: 1, TaskAttemptID:attempt_1463521955407_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A1403853B0D3429EAC6D6BD22BC4D2D8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:35,713 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:35,714 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:35,714 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:35,715 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521955407_0001_00_000002
2016-05-17 21:52:35,715 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:35,717 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521955407_0001_00_000002
2016-05-17 21:52:35,718 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:35,720 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:35,720 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:35,721 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521955407_0001_00_000002, launchTime=1463521955720
2016-05-17 21:52:35,721 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521955407_0001_00_000002
2016-05-17 21:52:35,722 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521955407_0001_1_01_000000_0] to container: [container_1463521955407_0001_00_000002]
2016-05-17 21:52:35,722 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:35,724 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521955407_0001_00_000002 given task: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,725 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521955407_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,726 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521955407_0001_1_01_000000_0] started. Is using containerId: [container_1463521955407_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:35,726 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0, startTime=1463521955726, containerId=container_1463521955407_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521955407_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:35,726 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:35,726 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:35,726 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:35 Starting to run new task attempt: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,727 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,727 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,728 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:35,728 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:35,728 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:35,728 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:35,729 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:35,729 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative, VertexName: F6AB05418EB14BBBA8BA4F876059A456, VertexParallelism: 1, TaskAttemptID:attempt_1463521955407_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A1403853B0D3429EAC6D6BD22BC4D2D8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:35,730 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:35,730 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,732 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:35,732 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,732 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521955407_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:35,734 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A1403853B0D3429EAC6D6BD22BC4D2D8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:35,734 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:35,733 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:35,733 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,735 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:35,735 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,734 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A1403853B0D3429EAC6D6BD22BC4D2D8
2016-05-17 21:52:35,737 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:35,737 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:35,737 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:35,738 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,738 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,737 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A1403853B0D3429EAC6D6BD22BC4D2D8
2016-05-17 21:52:35,738 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:35,749 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F6AB05418EB14BBBA8BA4F876059A456, ordinal: 1
2016-05-17 21:52:35,749 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6AB05418EB14BBBA8BA4F876059A456, mem on start (mb), free: 55, total: 555, max: 672
2016-05-17 21:52:35,749 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,750 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:35,750 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:35,752 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:35,753 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A1403853B0D3429EAC6D6BD22BC4D2D8
2016-05-17 21:52:35,753 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:35,753 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:35,753 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:35,753 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:35,753 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:35,753 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:35,754 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:35,754 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E1F5DCBD4B27474A9A5DFE6A8726B3EB, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:35,754 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A1403853B0D3429EAC6D6BD22BC4D2D8, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:35,754 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:35,754 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E1F5DCBD4B27474A9A5DFE6A8726B3EB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:35,754 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,754 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:35,755 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:35,755 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:35,756 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:35,756 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E1F5DCBD4B27474A9A5DFE6A8726B3EB with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:35,756 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:35,757 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:35,757 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:35,758 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,758 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:35,758 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,760 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521955407_0001_1_00_000000_0_10034, runDuration: 52342, ]
2016-05-17 21:52:35,762 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:35,764 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 22DB2FDF46894B7098BD96D37589992D
2016-05-17 21:52:35,764 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"], id: A1403853B0D3429EAC6D6BD22BC4D2D8
2016-05-17 21:52:35,765 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-17 21:52:35,765 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 22DB2FDF46894B7098BD96D37589992D, for 1 inputs
2016-05-17 21:52:35,766 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521955407_0001_1_00_000000_0_10034] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463521955766, TimeTaken=1, Rate=0.19 MB/s
2016-05-17 21:52:35,766 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,767 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:35,767 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E1F5DCBD4B27474A9A5DFE6A8726B3EB] #6 in 2ms
2016-05-17 21:52:35,768 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E1F5DCBD4B27474A9A5DFE6A8726B3EB] #6, status:false, isInterrupted:false
2016-05-17 21:52:35,768 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:35,769 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 21:52:35,769 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,769 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:35,769 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 21:52:35,770 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,770 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:35,770 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F6AB05418EB14BBBA8BA4F876059A456, all 1 inputs ready in: 00:00:00.005
2016-05-17 21:52:35,772 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@63a7de79; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@21ff0d0c
2016-05-17 21:52:35,775 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:35,775 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:35,776 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,776 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521955407_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:35,777 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,777 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6AB05418EB14BBBA8BA4F876059A456, mem on close (mb), free: 52, total: 555, max: 672
2016-05-17 21:52:35,777 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,778 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:35,778 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:35,779 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,779 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:35,779 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:35 Completed running task attempt: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,780 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:35,782 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:35,783 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521955771, Process_Duration=6, Process_End_Time=1463521955777, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 21:52:35,783 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:35,784 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,784 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521955407_0001_00_000002
2016-05-17 21:52:35,784 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0, startTime=1463521955726, finishTime=1463521955782, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955771, Process_Duration=6, Process_End_Time=1463521955777, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:35,784 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:35,784 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,784 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:35,785 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:35,785 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_FINISHED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, taskId=task_1463521955407_0001_1_01_000000, startTime=1463521955726, finishTime=1463521955784, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1463521955407_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955771, Process_Duration=6, Process_End_Time=1463521955777, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:35,785 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:35,785 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] : 1
2016-05-17 21:52:35,785 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,787 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:35,789 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_FINISHED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, vertexId=vertex_1463521955407_0001_1_01, initRequestedTime=1463521955591, initedTime=1463521955603, startRequestedTime=1463521955609, startedTime=1463521955609, finishTime=1463521955786, timeTaken=177, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955771, Process_Duration=6, Process_End_Time=1463521955777, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463521955726, firstTasksToStart=[ task_1463521955407_0001_1_01_000000 ], lastTaskFinishTime=1463521955782, lastTasksToFinish=[ task_1463521955407_0001_1_01_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521955407_0001_1_01_000000 ], longestDurationTasks=[ task_1463521955407_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:35,789 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:35,789 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:35,789 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521955407_0001_00_000002, stoppedTime=1463521955789, exitStatus=0
2016-05-17 21:52:35,789 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:35,790 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:35,790 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521955407_0001_1
2016-05-17 21:52:35,790 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521955407_0001_1
2016-05-17 21:52:35,790 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A1403853B0D3429EAC6D6BD22BC4D2D8 for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,789 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:35,795 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521955407_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:35,795 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521955407_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:35,795 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521955407_0001_00_000002, Exiting
2016-05-17 21:52:35,795 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521955407_0001_00_000002
2016-05-17 21:52:35,795 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521955407_0001_00_000002
2016-05-17 21:52:35,797 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521955407_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:35,797 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521955407_0001_00_000002 completed successfully
2016-05-17 21:52:35,799 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,801 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521955407_0001_1, queueSize=0
2016-05-17 21:52:35,802 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:35,803 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521955407_0001_1, startTime=1463521955588, finishTime=1463521955799, timeTaken=211, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1163919360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043911456, Process_Duration=8, Process_End_Time=2927043911464, Read_Duration=3, Tuples_Read=20, Tuples_Written=18, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 21:52:35,803 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521955407_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:35,803 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521955407_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:35,804 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521955407_0001_00_000002
2016-05-17 21:52:35,804 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:35 Completed Dag: dag_1463521955407_0001_1
2016-05-17 21:52:35,805 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521955407_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:35,805 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521955407_0001_1_post
2016-05-17 21:52:35,805 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,806 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:35,811 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463521955407_0001
2016-05-17 21:52:35,814 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:35,814 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:35,814 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:35,814 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:35,814 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:35,815 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:35,815 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:35,816 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:35,816 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:35,817 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:35,817 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:35,817 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:35,818 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:35,822 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001
2016-05-17 21:52:35,823 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative/_temporary
2016-05-17 21:52:35,823 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.437
2016-05-17 21:52:35,879 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-17 21:52:36,006 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:36,007 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:36,062 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-17 21:52:36,073 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-05-17 21:52:36,074 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:36,074 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:36,074 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:36,091 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 21:52:36,107 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-17 21:52:36,167 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:36,168 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:36,168 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-17 21:52:36,168 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:36,169 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:36,169 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:36,172 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36,172 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:36,191 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:36,196 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:36,197 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:36,197 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:36,206 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001 doesn't exist and is created
2016-05-17 21:52:36,223 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:36,223 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:36,223 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001
2016-05-17 21:52:36,227 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521956197_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:36,238 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:36,239 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:36,241 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:36,241 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:36,241 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001/localmode-log-dir
2016-05-17 21:52:36,242 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001/localmode-log-dir/history.txt.appattempt_1463521956197_0001_000000, maxErrors=10
2016-05-17 21:52:36,242 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:36,242 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521956197_0001_000000, appSubmitTime=1463521956227, launchTime=1463521956227
2016-05-17 21:52:36,246 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:36,248 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:36,250 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:36,250 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:36,255 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53079
2016-05-17 21:52:36,256 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:36,260 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521956197_0001_000000, startTime=1463521956260
2016-05-17 21:52:36,260 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:36,260 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:36,323 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:36,323 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:36,323 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463521956197_0001, dagName=[19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36,343 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36,344 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521956197_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001/localmode-log-dir/dag_1463521956197_0001_1.dot
2016-05-17 21:52:36,345 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001/localmode-log-dir/dag_1463521956197_0001_1-tez-dag.pb.txt
2016-05-17 21:52:36,360 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521956197_0001_1
2016-05-17 21:52:36,360 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,361 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36 Running Dag: dag_1463521956197_0001_1
2016-05-17 21:52:36,380 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521956197_0001_1, submitTime=1463521956343
2016-05-17 21:52:36,380 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:36,381 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,381 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:36,382 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:36,383 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521956197_0001_1, initTime=1463521956380
2016-05-17 21:52:36,383 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956197_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:36,383 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_STARTED]: dagID=dag_1463521956197_0001_1, startTime=1463521956383
2016-05-17 21:52:36,383 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:36,384 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956197_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:36,384 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FF21EEC1C04942F492745C179A05836E : {F0420273451947A19C658D80B1A9CE89={InputName=F0420273451947A19C658D80B1A9CE89}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:36,384 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F0420273451947A19C658D80B1A9CE89, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:36,384 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,384 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,384 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,385 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]. Starting root input initializers: 1
2016-05-17 21:52:36,385 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:36,385 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:36,386 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,388 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F0420273451947A19C658D80B1A9CE89 on vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,390 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:36,390 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F0420273451947A19C658D80B1A9CE89 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:36,392 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:36,393 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463521956197_0001, dagName=[19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36,393 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:36,394 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,394 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,394 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,394 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CB011DFD10FD4ED48B938E6E5F207A03, vertexId=vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:36,394 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CB011DFD10FD4ED48B938E6E5F207A03, vertexId=vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,398 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CB011DFD10FD4ED48B938E6E5F207A03, vertexName=A04EAE2F31B3429787C5735E432239E1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:36,399 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CB011DFD10FD4ED48B938E6E5F207A03, vertexId=vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,401 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:36,401 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A04EAE2F31B3429787C5735E432239E1, vertexId=vertex_1463521956197_0001_1_01, initRequestedTime=1463521956386, initedTime=1463521956401, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:36,402 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:36,404 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:36,407 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:36,410 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F0420273451947A19C658D80B1A9CE89 on vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,410 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:36,410 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F0420273451947A19C658D80B1A9CE89=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:36,411 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] parallelism set to 1
2016-05-17 21:52:36,411 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:36,412 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521956197_0001_1_00, eventCount=1
2016-05-17 21:52:36,413 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:36,414 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF21EEC1C04942F492745C179A05836E, vertexId=vertex_1463521956197_0001_1_00, initRequestedTime=1463521956384, initedTime=1463521956413, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:36,414 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] with distanceFromRoot: 0
2016-05-17 21:52:36,414 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:36,415 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,415 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,416 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:36,416 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521956197_0001_1_00, startRequestedTime=1463521956415, startedTime=1463521956415
2016-05-17 21:52:36,416 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:36,417 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:36,418 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_STARTED]: vertexName=FF21EEC1C04942F492745C179A05836E, taskId=task_1463521956197_0001_1_00_000000, scheduledTime=1463521956417, launchTime=1463521956417
2016-05-17 21:52:36,418 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:36,418 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521956197_0001_1_00 for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] numStartedSources: 1 numSources: 1
2016-05-17 21:52:36,419 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] with distanceFromRoot: 1
2016-05-17 21:52:36,419 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,420 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FF21EEC1C04942F492745C179A05836E in vertex: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,420 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A04EAE2F31B3429787C5735E432239E1 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:36,420 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:36,421 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521956197_0001_1_01, startRequestedTime=1463521956419, startedTime=1463521956419
2016-05-17 21:52:36,421 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:36,422 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult, VertexName: FF21EEC1C04942F492745C179A05836E, VertexParallelism: 1, TaskAttemptID:attempt_1463521956197_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0420273451947A19C658D80B1A9CE89, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A04EAE2F31B3429787C5735E432239E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:36,422 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:36,422 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:36,423 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521956197_0001, with dag id: dag_1463521956197_0001_1
2016-05-17 21:52:36,423 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:36,424 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:36,424 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:36,425 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521956197_0001_00_000001
2016-05-17 21:52:36,425 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:36,428 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521956197_0001_00_000001
2016-05-17 21:52:36,428 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:36,429 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:36,429 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:36,430 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521956197_0001_00_000001, launchTime=1463521956429
2016-05-17 21:52:36,431 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956197_0001_00_000001
2016-05-17 21:52:36,432 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521956197_0001_1_00_000000_0] to container: [container_1463521956197_0001_00_000001]
2016-05-17 21:52:36,432 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:36,433 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521956197_0001_1_00_000000_0] started. Is using containerId: [container_1463521956197_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:36,433 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:36,433 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF21EEC1C04942F492745C179A05836E, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0, startTime=1463521956432, containerId=container_1463521956197_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521956197_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:36,433 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:36,433 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:36,434 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521956197_0001_00_000001 given task: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,434 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956197_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36 Starting to run new task attempt: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,434 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,434 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,435 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:36,435 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:36,435 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:36,435 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:36,435 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:36,435 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult, VertexName: FF21EEC1C04942F492745C179A05836E, VertexParallelism: 1, TaskAttemptID:attempt_1463521956197_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0420273451947A19C658D80B1A9CE89, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A04EAE2F31B3429787C5735E432239E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:36,435 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:36,436 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,436 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:36,436 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,437 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,440 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521956197_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:36,440 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:36,442 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A04EAE2F31B3429787C5735E432239E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:36,442 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:36,442 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,447 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:36,447 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,447 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,448 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F0420273451947A19C658D80B1A9CE89, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:36,448 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:36,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,449 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,449 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF21EEC1C04942F492745C179A05836E, ordinal: 0
2016-05-17 21:52:36,450 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF21EEC1C04942F492745C179A05836E, mem on start (mb), free: 176, total: 554, max: 672
2016-05-17 21:52:36,450 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,450 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:36,450 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:36,453 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:36,454 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,454 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,455 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:36,455 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:36,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:36,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:36,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:36,455 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:36,456 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:36,456 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A04EAE2F31B3429787C5735E432239E1, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:36,456 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F0420273451947A19C658D80B1A9CE89, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:36,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:36,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,456 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:36,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,457 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F0420273451947A19C658D80B1A9CE89 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:36,457 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:36,457 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:36,457 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:36,458 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,459 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:36,461 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:36,461 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7e214d47
2016-05-17 21:52:36,462 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,462 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 49A0EB06D5494EE09C0F1CC20E49081F
2016-05-17 21:52:36,463 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 49A0EB06D5494EE09C0F1CC20E49081F
2016-05-17 21:52:36,463 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7e214d47
2016-05-17 21:52:36,463 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:36,463 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:36,465 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@44f0adaf; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@151adf85; comparator=cascading.tuple.hadoop.util.TupleComparator@17489f96; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:36,465 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:36,465 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:36,466 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:36,479 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:36,479 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:36,479 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:36,479 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:36,479 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:36,555 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:36,556 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:36,556 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF21EEC1C04942F492745C179A05836E, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:36,559 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,559 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF21EEC1C04942F492745C179A05836E, mem on close (mb), free: 68, total: 554, max: 672
2016-05-17 21:52:36,559 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:36,559 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:36,560 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 22; bufvoid = 104857600
2016-05-17 21:52:36,560 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2016-05-17 21:52:36,563 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:36 Completed running task attempt: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,566 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:36,567 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521956197_0001_1_00, eventCount=1
2016-05-17 21:52:36,567 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:36,567 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF21EEC1C04942F492745C179A05836E, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0, startTime=1463521956432, finishTime=1463521956567, timeTaken=135, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956556, Process_Duration=3, Process_End_Time=1463521956559, Read_Duration=1, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:36,568 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:36,568 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:36,568 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,569 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:36,569 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_FINISHED]: vertexName=FF21EEC1C04942F492745C179A05836E, taskId=task_1463521956197_0001_1_00_000000, startTime=1463521956432, finishTime=1463521956568, timeTaken=136, status=SUCCEEDED, successfulAttemptID=attempt_1463521956197_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956556, Process_Duration=3, Process_End_Time=1463521956559, Read_Duration=1, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:36,569 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:36,569 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] : 1
2016-05-17 21:52:36,570 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:36,571 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_FINISHED]: vertexName=FF21EEC1C04942F492745C179A05836E, vertexId=vertex_1463521956197_0001_1_00, initRequestedTime=1463521956384, initedTime=1463521956413, startRequestedTime=1463521956415, startedTime=1463521956415, finishTime=1463521956570, timeTaken=155, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956556, Process_Duration=3, Process_End_Time=1463521956559, Read_Duration=1, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521956432, firstTasksToStart=[ task_1463521956197_0001_1_00_000000 ], lastTaskFinishTime=1463521956567, lastTasksToFinish=[ task_1463521956197_0001_1_00_000000 ], minTaskDuration=135, maxTaskDuration=135, avgTaskDuration=135.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521956197_0001_1_00_000000 ], longestDurationTasks=[ task_1463521956197_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:36,571 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:36,572 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:36,572 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521956556, Process_Duration=3, Process_End_Time=1463521956559, Read_Duration=1, Tuples_Read=5, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:36,572 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:36,572 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521956197_0001_00_000001, stoppedTime=1463521956572, exitStatus=0
2016-05-17 21:52:36,574 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:36,573 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:36,574 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,574 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] attempt: attempt_1463521956197_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:36,574 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956197_0001_00_000001
2016-05-17 21:52:36,575 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521956197_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:36,575 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956197_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:36,574 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,575 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521956197_0001_00_000001, Exiting
2016-05-17 21:52:36,575 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521956197_0001_00_000001
2016-05-17 21:52:36,575 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521956197_0001_00_000001
2016-05-17 21:52:36,575 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,576 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:36,577 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521956197_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:36,578 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:36,578 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521956197_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:36,578 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_STARTED]: vertexName=A04EAE2F31B3429787C5735E432239E1, taskId=task_1463521956197_0001_1_01_000000, scheduledTime=1463521956578, launchTime=1463521956578
2016-05-17 21:52:36,578 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521956197_0001_00_000001 completed successfully
2016-05-17 21:52:36,578 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:36,579 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:36,579 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,579 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:36,579 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult, VertexName: A04EAE2F31B3429787C5735E432239E1, VertexParallelism: 1, TaskAttemptID:attempt_1463521956197_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF21EEC1C04942F492745C179A05836E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CB011DFD10FD4ED48B938E6E5F207A03, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:36,579 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:36,580 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:36,580 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521956197_0001_00_000002
2016-05-17 21:52:36,581 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:36,582 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521956197_0001_00_000002
2016-05-17 21:52:36,583 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:36,585 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:36,585 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:36,585 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521956197_0001_00_000002, launchTime=1463521956585
2016-05-17 21:52:36,586 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956197_0001_00_000002
2016-05-17 21:52:36,587 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521956197_0001_1_01_000000_0] to container: [container_1463521956197_0001_00_000002]
2016-05-17 21:52:36,587 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:36,588 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521956197_0001_00_000002 given task: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,588 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956197_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,588 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521956197_0001_1_01_000000_0] started. Is using containerId: [container_1463521956197_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:36,589 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A04EAE2F31B3429787C5735E432239E1, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0, startTime=1463521956588, containerId=container_1463521956197_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521956197_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:36,589 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:36,589 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:36,589 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:36 Starting to run new task attempt: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,590 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,591 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,591 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:36,591 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:36,592 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:36,592 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:36,592 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:36,592 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult, VertexName: A04EAE2F31B3429787C5735E432239E1, VertexParallelism: 1, TaskAttemptID:attempt_1463521956197_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF21EEC1C04942F492745C179A05836E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CB011DFD10FD4ED48B938E6E5F207A03, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:36,593 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:36,593 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:36,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,597 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521956197_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:36,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FF21EEC1C04942F492745C179A05836E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:36,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CB011DFD10FD4ED48B938E6E5F207A03, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:36,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:36,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CB011DFD10FD4ED48B938E6E5F207A03
2016-05-17 21:52:36,599 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:36,599 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CB011DFD10FD4ED48B938E6E5F207A03
2016-05-17 21:52:36,600 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:36,600 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,604 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:36,604 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:36,604 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,605 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:36,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,614 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:36,614 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CB011DFD10FD4ED48B938E6E5F207A03
2016-05-17 21:52:36,616 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A04EAE2F31B3429787C5735E432239E1, ordinal: 1
2016-05-17 21:52:36,616 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A04EAE2F31B3429787C5735E432239E1, mem on start (mb), free: 62, total: 554, max: 672
2016-05-17 21:52:36,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:36,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:36,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:36,616 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:36,618 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:36,618 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:36,618 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:36,618 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:36,618 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:36,619 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CB011DFD10FD4ED48B938E6E5F207A03, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:36,619 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FF21EEC1C04942F492745C179A05836E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:36,619 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:36,619 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FF21EEC1C04942F492745C179A05836E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:36,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:36,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:36,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,621 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:36,622 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:36,622 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FF21EEC1C04942F492745C179A05836E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:36,622 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:36,622 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:36,622 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:36,623 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,624 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:36,625 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,625 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521956197_0001_1_00_000000_0_10039, runDuration: 122339, ]
2016-05-17 21:52:36,626 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521956197_0001_1_00_000000_0_10039] to DISK_DIRECT, CompressedSize=36, DecompressedSize=32,EndTime=1463521956626, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:52:36,627 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,627 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:36,627 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [FF21EEC1C04942F492745C179A05836E] #7 in 1ms
2016-05-17 21:52:36,627 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FF21EEC1C04942F492745C179A05836E] #7, status:false, isInterrupted:false
2016-05-17 21:52:36,628 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:36,628 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 36 bytes from disk
2016-05-17 21:52:36,628 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:36,628 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:36,630 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2016-05-17 21:52:36,630 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,631 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:36,630 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:36,632 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 49A0EB06D5494EE09C0F1CC20E49081F
2016-05-17 21:52:36,632 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"], id: CB011DFD10FD4ED48B938E6E5F207A03
2016-05-17 21:52:36,633 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-17 21:52:36,634 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 49A0EB06D5494EE09C0F1CC20E49081F, for 1 inputs
2016-05-17 21:52:36,634 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A04EAE2F31B3429787C5735E432239E1, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:36,638 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@56f47006; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@49e6c683
2016-05-17 21:52:36,639 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:36,640 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:36,640 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,641 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521956197_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:36,642 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,643 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A04EAE2F31B3429787C5735E432239E1, mem on close (mb), free: 58, total: 554, max: 672
2016-05-17 21:52:36,643 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,644 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:36,645 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:36,645 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:36,645 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:36,646 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:36 Completed running task attempt: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,648 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:36,649 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521956634, Process_Duration=8, Process_End_Time=1463521956642, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=2]]
2016-05-17 21:52:36,649 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:36,650 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:36,650 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,650 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956197_0001_00_000002
2016-05-17 21:52:36,649 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A04EAE2F31B3429787C5735E432239E1, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0, startTime=1463521956588, finishTime=1463521956649, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956634, Process_Duration=8, Process_End_Time=1463521956642, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-05-17 21:52:36,650 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:36,650 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:36,651 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,651 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:36,651 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_FINISHED]: vertexName=A04EAE2F31B3429787C5735E432239E1, taskId=task_1463521956197_0001_1_01_000000, startTime=1463521956588, finishTime=1463521956651, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463521956197_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956634, Process_Duration=8, Process_End_Time=1463521956642, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-05-17 21:52:36,652 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:36,652 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] : 1
2016-05-17 21:52:36,652 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,653 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:36,653 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_FINISHED]: vertexName=A04EAE2F31B3429787C5735E432239E1, vertexId=vertex_1463521956197_0001_1_01, initRequestedTime=1463521956386, initedTime=1463521956401, startRequestedTime=1463521956419, startedTime=1463521956419, finishTime=1463521956652, timeTaken=233, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956634, Process_Duration=8, Process_End_Time=1463521956642, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2, vertexStats=firstTaskStartTime=1463521956588, firstTasksToStart=[ task_1463521956197_0001_1_01_000000 ], lastTaskFinishTime=1463521956649, lastTasksToFinish=[ task_1463521956197_0001_1_01_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521956197_0001_1_01_000000 ], longestDurationTasks=[ task_1463521956197_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:36,653 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:36,654 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:36,654 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521956197_0001_00_000002, stoppedTime=1463521956654, exitStatus=0
2016-05-17 21:52:36,654 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:36,654 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:36,654 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:36,654 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521956197_0001_1
2016-05-17 21:52:36,655 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521956197_0001_1
2016-05-17 21:52:36,656 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CB011DFD10FD4ED48B938E6E5F207A03 for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,660 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521956197_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:36,660 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956197_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:36,660 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521956197_0001_00_000002, Exiting
2016-05-17 21:52:36,661 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521956197_0001_00_000002
2016-05-17 21:52:36,661 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521956197_0001_00_000002
2016-05-17 21:52:36,662 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521956197_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:36,662 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521956197_0001_00_000002 completed successfully
2016-05-17 21:52:36,664 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,667 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521956197_0001_1, queueSize=0
2016-05-17 21:52:36,667 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:36,668 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521956197_0001_1, startTime=1463521956383, finishTime=1463521956665, timeTaken=282, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=130, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1162870784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043913190, Process_Duration=11, Process_End_Time=2927043913201, Read_Duration=1, Tuples_Read=7, Tuples_Written=4, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=2
2016-05-17 21:52:36,668 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521956197_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:36,668 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956197_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:36,668 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:36,669 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521956197_0001_00_000002
2016-05-17 21:52:36 Completed Dag: dag_1463521956197_0001_1
2016-05-17 21:52:36,670 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521956197_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:36,670 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521956197_0001_1_post
2016-05-17 21:52:36,670 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,671 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:36,676 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463521956197_0001
2016-05-17 21:52:36,678 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:36,678 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:36,679 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:36,679 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:36,679 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:36,679 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:36,680 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:36,680 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:36,680 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:36,680 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:36,681 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:36,681 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:36,681 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:36,686 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001
2016-05-17 21:52:36,687 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult/_temporary
2016-05-17 21:52:36,687 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.519
2016-05-17 21:52:36,723 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-17 21:52:36,758 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:36,758 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:36,796 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:52:36,804 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 21:52:36,805 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:36,805 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:36,806 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:36,822 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:36,824 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-17 21:52:36,834 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:36,834 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:36,834 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-17 21:52:36,835 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:36,835 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:36,836 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:36,837 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:36,837 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:36,843 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:36,843 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:36,856 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:36,861 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:36,861 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:36,861 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:36,872 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001 doesn't exist and is created
2016-05-17 21:52:36,889 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:36,890 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:36,890 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001
2016-05-17 21:52:36,893 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521956861_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:36,903 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:36,904 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:36,906 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:36,906 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:36,906 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001/localmode-log-dir
2016-05-17 21:52:36,907 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001/localmode-log-dir/history.txt.appattempt_1463521956861_0001_000000, maxErrors=10
2016-05-17 21:52:36,907 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:36,907 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521956861_0001_000000, appSubmitTime=1463521956893, launchTime=1463521956893
2016-05-17 21:52:36,913 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36648
2016-05-17 21:52:36,916 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:36,916 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:36,917 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:36,918 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:36,919 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:36,921 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521956861_0001_000000, startTime=1463521956921
2016-05-17 21:52:36,921 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:36,921 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:36,990 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:36,990 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:36,990 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463521956861_0001, dagName=[66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:37,006 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:37,008 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521956861_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001/localmode-log-dir/dag_1463521956861_0001_1.dot
2016-05-17 21:52:37,008 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001/localmode-log-dir/dag_1463521956861_0001_1-tez-dag.pb.txt
2016-05-17 21:52:37,017 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521956861_0001_1
2016-05-17 21:52:37,017 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,017 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:37 Running Dag: dag_1463521956861_0001_1
2016-05-17 21:52:37,029 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521956861_0001_1, submitTime=1463521957006
2016-05-17 21:52:37,029 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:37,030 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D4A883C13A1A4646A28E43F824B680AB
2016-05-17 21:52:37,030 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:37,031 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521956861_0001_1, initTime=1463521957030
2016-05-17 21:52:37,031 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:37,032 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956861_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:37,032 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_STARTED]: dagID=dag_1463521956861_0001_1, startTime=1463521957032
2016-05-17 21:52:37,032 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:37,033 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:37,033 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956861_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:37,033 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D4A883C13A1A4646A28E43F824B680AB : {372E46651972455886F26311FE2E210F={InputName=372E46651972455886F26311FE2E210F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:37,033 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 372E46651972455886F26311FE2E210F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:37,033 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,034 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,034 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,034 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463521956861_0001, dagName=[66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:37,034 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]. Starting root input initializers: 1
2016-05-17 21:52:37,035 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:37,035 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 372E46651972455886F26311FE2E210F on vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,037 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:37,037 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 372E46651972455886F26311FE2E210F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:37,038 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:37,038 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:37,040 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:37,041 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 372E46651972455886F26311FE2E210F on vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,041 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:37,041 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {372E46651972455886F26311FE2E210F=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:37,042 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] parallelism set to 1
2016-05-17 21:52:37,042 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:37,042 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521956861_0001_1_00, eventCount=1
2016-05-17 21:52:37,042 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,042 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A3CAE8CF934D4F518F3AA3431C98869A, vertexId=vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:37,043 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A3CAE8CF934D4F518F3AA3431C98869A, vertexId=vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,044 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A3CAE8CF934D4F518F3AA3431C98869A, vertexName=D4A883C13A1A4646A28E43F824B680AB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:37,045 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A3CAE8CF934D4F518F3AA3431C98869A, vertexId=vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,046 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:37,047 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, vertexId=vertex_1463521956861_0001_1_00, initRequestedTime=1463521957033, initedTime=1463521957046, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:37,047 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] with distanceFromRoot: 0
2016-05-17 21:52:37,047 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:37,047 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D4A883C13A1A4646A28E43F824B680AB
2016-05-17 21:52:37,047 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,048 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:37,048 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521956861_0001_1_00, startRequestedTime=1463521957047, startedTime=1463521957047
2016-05-17 21:52:37,048 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:37,048 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:37,049 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:TASK_STARTED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, taskId=task_1463521956861_0001_1_00_000000, scheduledTime=1463521957048, launchTime=1463521957048
2016-05-17 21:52:37,049 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956861_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:37,049 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,049 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall, VertexName: D4A883C13A1A4646A28E43F824B680AB, VertexParallelism: 1, TaskAttemptID:attempt_1463521956861_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=372E46651972455886F26311FE2E210F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A3CAE8CF934D4F518F3AA3431C98869A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:37,050 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956861_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:37,050 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:37,050 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:37,051 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:37,051 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:37,052 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521956861_0001_00_000001
2016-05-17 21:52:37,052 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:37,053 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521956861_0001_00_000001
2016-05-17 21:52:37,053 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:37,054 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:37,054 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:37,055 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521956861_0001, with dag id: dag_1463521956861_0001_1
2016-05-17 21:52:37,055 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521956861_0001_00_000001, launchTime=1463521957054
2016-05-17 21:52:37,056 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956861_0001_00_000001
2016-05-17 21:52:37,058 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521956861_0001_1_00_000000_0] to container: [container_1463521956861_0001_00_000001]
2016-05-17 21:52:37,058 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:37,060 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521956861_0001_1_00_000000_0] started. Is using containerId: [container_1463521956861_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:37,061 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0, startTime=1463521957059, containerId=container_1463521956861_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521956861_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:37,061 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956861_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:37,061 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956861_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:37,062 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:37,062 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521956861_0001_00_000001 given task: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,062 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956861_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37 Starting to run new task attempt: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,063 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,063 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,063 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:37,063 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:37,063 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:37,063 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:37,063 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:37,064 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall, VertexName: D4A883C13A1A4646A28E43F824B680AB, VertexParallelism: 1, TaskAttemptID:attempt_1463521956861_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=372E46651972455886F26311FE2E210F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A3CAE8CF934D4F518F3AA3431C98869A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:37,064 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:37,064 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,064 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:37,065 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,067 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521956861_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:37,067 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=372E46651972455886F26311FE2E210F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:37,068 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:37,068 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,069 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,068 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A3CAE8CF934D4F518F3AA3431C98869A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:37,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:37,070 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:37,072 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:37,073 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A3CAE8CF934D4F518F3AA3431C98869A
2016-05-17 21:52:37,074 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:37,074 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A3CAE8CF934D4F518F3AA3431C98869A
2016-05-17 21:52:37,083 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D4A883C13A1A4646A28E43F824B680AB, ordinal: 0
2016-05-17 21:52:37,083 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4A883C13A1A4646A28E43F824B680AB, mem on start (mb), free: 204, total: 554, max: 672
2016-05-17 21:52:37,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:37,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:37,087 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:37,088 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A3CAE8CF934D4F518F3AA3431C98869A
2016-05-17 21:52:37,088 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:37,088 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:37,089 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:37,090 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:37,090 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:37,090 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:37,090 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:37,091 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:37,091 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 372E46651972455886F26311FE2E210F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:37,091 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A3CAE8CF934D4F518F3AA3431C98869A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:37,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:37,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 372E46651972455886F26311FE2E210F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:37,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,092 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:37,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,093 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:37,093 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:37,093 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:37,093 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,096 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:37,096 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,096 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:37,096 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"], id: A3CAE8CF934D4F518F3AA3431C98869A
2016-05-17 21:52:37,097 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-17 21:52:37,097 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:37,097 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:37,097 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7e5bf9e7
2016-05-17 21:52:37,099 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7e5bf9e7
2016-05-17 21:52:37,099 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:37,100 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:37,100 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D4A883C13A1A4646A28E43F824B680AB, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:37,102 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:37,102 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:37,102 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,102 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521956861_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:37,103 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,103 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4A883C13A1A4646A28E43F824B680AB, mem on close (mb), free: 203, total: 554, max: 672
2016-05-17 21:52:37 Completed running task attempt: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,104 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:37,105 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0, startTime=1463521957059, finishTime=1463521957104, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:37,105 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956861_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:37,105 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,105 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:TASK_FINISHED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, taskId=task_1463521956861_0001_1_00_000000, startTime=1463521957059, finishTime=1463521957105, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463521956861_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:37,106 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956861_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:37,106 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] : 1
2016-05-17 21:52:37,106 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,106 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:37,107 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:37,107 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:37,108 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:VERTEX_FINISHED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, vertexId=vertex_1463521956861_0001_1_00, initRequestedTime=1463521957033, initedTime=1463521957046, startRequestedTime=1463521957047, startedTime=1463521957047, finishTime=1463521957106, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463521957059, firstTasksToStart=[ task_1463521956861_0001_1_00_000000 ], lastTaskFinishTime=1463521957104, lastTasksToFinish=[ task_1463521956861_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521956861_0001_1_00_000000 ], longestDurationTasks=[ task_1463521956861_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:37,108 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:37,108 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:37,109 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:37,109 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:37,109 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521956861_0001_1
2016-05-17 21:52:37,109 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521956861_0001_1
2016-05-17 21:52:37,109 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A3CAE8CF934D4F518F3AA3431C98869A for vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,112 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 21:52:37,112 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:37,113 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,113 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956861_0001_00_000001
2016-05-17 21:52:37,117 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521956861_0001_1, queueSize=0
2016-05-17 21:52:37,118 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:37,118 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521956861_0001_1, startTime=1463521957032, finishTime=1463521957116, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:37,118 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521956861_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:37,119 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956861_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:37,119 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:37,119 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521956861_0001_00_000001, stoppedTime=1463521957119, exitStatus=0
2016-05-17 21:52:37,120 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:37 Completed Dag: dag_1463521956861_0001_1
2016-05-17 21:52:37,121 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521956861_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:37,121 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521956861_0001_1_post
2016-05-17 21:52:37,121 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,122 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:37,122 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521956861_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:37,122 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:37,123 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521956861_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:37,123 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956861_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:37,123 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521956861_0001_00_000001, Exiting
2016-05-17 21:52:37,124 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521956861_0001_00_000001
2016-05-17 21:52:37,124 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521956861_0001_00_000001
2016-05-17 21:52:37,127 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521956861_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:37,127 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521956861_0001_00_000001 completed successfully
2016-05-17 21:52:37,128 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:37,127 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463521956861_0001
2016-05-17 21:52:37,130 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:37,130 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:37,130 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:37,130 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:37,130 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:37,131 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:37,131 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:37,131 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:37,131 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:37,131 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:37,131 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:37,132 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:37,133 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:37,138 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001
2016-05-17 21:52:37,138 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall/_temporary
2016-05-17 21:52:37,139 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.305
2016-05-17 21:52:37,175 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-17 21:52:37,204 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:37,205 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:37,247 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 21:52:37,260 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 21:52:37,260 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:37,261 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:37,262 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:37,277 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 21:52:37,285 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-17 21:52:37,336 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:37,336 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:37,336 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-17 21:52:37,336 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:37,336 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:37,336 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:37,337 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37,337 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:37,350 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:37,354 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:37,355 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:37,355 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:37,364 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001 doesn't exist and is created
2016-05-17 21:52:37,380 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:37,381 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:37,381 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001
2016-05-17 21:52:37,386 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521957355_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:37,396 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:37,397 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:37,400 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:37,401 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:37,401 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001/localmode-log-dir
2016-05-17 21:52:37,402 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001/localmode-log-dir/history.txt.appattempt_1463521957355_0001_000000, maxErrors=10
2016-05-17 21:52:37,402 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:37,403 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521957355_0001_000000, appSubmitTime=1463521957385, launchTime=1463521957385
2016-05-17 21:52:37,407 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:37,409 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:37,410 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58009
2016-05-17 21:52:37,412 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:37,412 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:37,413 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:37,417 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521957355_0001_000000, startTime=1463521957417
2016-05-17 21:52:37,417 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:37,417 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:37,481 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:37,481 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:37,481 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463521957355_0001, dagName=[41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37,498 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37,499 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521957355_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001/localmode-log-dir/dag_1463521957355_0001_1.dot
2016-05-17 21:52:37,500 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001/localmode-log-dir/dag_1463521957355_0001_1-tez-dag.pb.txt
2016-05-17 21:52:37,513 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521957355_0001_1
2016-05-17 21:52:37,513 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,513 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37 Running Dag: dag_1463521957355_0001_1
2016-05-17 21:52:37,526 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521957355_0001_1, submitTime=1463521957498
2016-05-17 21:52:37,526 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:37,528 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,528 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:37,528 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521957355_0001_1, initTime=1463521957526
2016-05-17 21:52:37,529 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521957355_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:37,529 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:37,530 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_STARTED]: dagID=dag_1463521957355_0001_1, startTime=1463521957529
2016-05-17 21:52:37,530 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:37,531 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521957355_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:37,531 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FD7472811FE94572A8A815EDDB0A8842 : {C66A7A28B69B4DEF8D8CE1EB85C45DBE={InputName=C66A7A28B69B4DEF8D8CE1EB85C45DBE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:37,530 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:37,531 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463521957355_0001, dagName=[41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37,531 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C66A7A28B69B4DEF8D8CE1EB85C45DBE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:37,532 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,532 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,532 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,532 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]. Starting root input initializers: 1
2016-05-17 21:52:37,532 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:37,533 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,534 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C66A7A28B69B4DEF8D8CE1EB85C45DBE on vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,535 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:37,536 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:37,536 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,536 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,537 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,537 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=15C86E44C79B4390A54B7D031375A2B9, vertexId=vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:37,537 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=15C86E44C79B4390A54B7D031375A2B9, vertexId=vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,536 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C66A7A28B69B4DEF8D8CE1EB85C45DBE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:37,539 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:37,539 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:37,540 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=15C86E44C79B4390A54B7D031375A2B9, vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:37,540 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=15C86E44C79B4390A54B7D031375A2B9, vertexId=vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,541 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:37,542 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C66A7A28B69B4DEF8D8CE1EB85C45DBE on vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,542 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:37,542 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, vertexId=vertex_1463521957355_0001_1_01, initRequestedTime=1463521957533, initedTime=1463521957542, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:37,543 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:37,543 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:37,543 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C66A7A28B69B4DEF8D8CE1EB85C45DBE=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:37,543 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] parallelism set to 1
2016-05-17 21:52:37,544 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:37,544 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521957355_0001_1_00, eventCount=1
2016-05-17 21:52:37,544 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, vertexId=vertex_1463521957355_0001_1_00, initRequestedTime=1463521957531, initedTime=1463521957544, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:37,545 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] with distanceFromRoot: 0
2016-05-17 21:52:37,544 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:37,545 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:37,545 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,546 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,546 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:37,546 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521957355_0001_1_00, startRequestedTime=1463521957545, startedTime=1463521957545
2016-05-17 21:52:37,547 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:37,547 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:37,547 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_STARTED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, taskId=task_1463521957355_0001_1_00_000000, scheduledTime=1463521957547, launchTime=1463521957547
2016-05-17 21:52:37,548 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:37,548 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521957355_0001_1_00 for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] numStartedSources: 1 numSources: 1
2016-05-17 21:52:37,548 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] with distanceFromRoot: 1
2016-05-17 21:52:37,549 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,549 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FD7472811FE94572A8A815EDDB0A8842 in vertex: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,549 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E4FB3BD594E64AE2AAF803F99DE6B0E7 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:37,550 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:37,550 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521957355_0001_1_01, startRequestedTime=1463521957549, startedTime=1463521957549
2016-05-17 21:52:37,550 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:37,551 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown, VertexName: FD7472811FE94572A8A815EDDB0A8842, VertexParallelism: 1, TaskAttemptID:attempt_1463521957355_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C66A7A28B69B4DEF8D8CE1EB85C45DBE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:37,551 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:37,551 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:37,551 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521957355_0001, with dag id: dag_1463521957355_0001_1
2016-05-17 21:52:37,553 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:37,554 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:37,555 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:37,556 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521957355_0001_00_000001
2016-05-17 21:52:37,556 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:37,558 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521957355_0001_00_000001
2016-05-17 21:52:37,558 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:37,559 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:37,559 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:37,559 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521957355_0001_00_000001, launchTime=1463521957559
2016-05-17 21:52:37,560 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521957355_0001_00_000001
2016-05-17 21:52:37,561 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521957355_0001_1_00_000000_0] to container: [container_1463521957355_0001_00_000001]
2016-05-17 21:52:37,561 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:37,562 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521957355_0001_1_00_000000_0] started. Is using containerId: [container_1463521957355_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:37,562 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0, startTime=1463521957561, containerId=container_1463521957355_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521957355_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:37,562 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:37,562 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:37,562 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:37,562 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521957355_0001_00_000001 given task: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,562 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521957355_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37 Starting to run new task attempt: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,563 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,563 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,563 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:37,563 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:37,563 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:37,563 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:37,564 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:37,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown, VertexName: FD7472811FE94572A8A815EDDB0A8842, VertexParallelism: 1, TaskAttemptID:attempt_1463521957355_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C66A7A28B69B4DEF8D8CE1EB85C45DBE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:37,564 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:37,564 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,564 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:37,565 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,565 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521957355_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:37,566 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C66A7A28B69B4DEF8D8CE1EB85C45DBE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:37,566 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,566 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:37,566 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,566 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,569 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:37,569 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,569 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:37,569 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,568 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:37,571 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:37,571 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,573 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:37,573 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,575 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,575 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FD7472811FE94572A8A815EDDB0A8842, ordinal: 0
2016-05-17 21:52:37,575 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD7472811FE94572A8A815EDDB0A8842, mem on start (mb), free: 116, total: 554, max: 672
2016-05-17 21:52:37,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:37,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:37,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:37,577 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:37,577 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:37,578 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:37,578 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:37,578 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:37,578 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:37,579 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C66A7A28B69B4DEF8D8CE1EB85C45DBE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:37,579 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E4FB3BD594E64AE2AAF803F99DE6B0E7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:37,579 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:37,579 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C66A7A28B69B4DEF8D8CE1EB85C45DBE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:37,580 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:37,580 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:37,580 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,581 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:37,581 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,581 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:37,582 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,585 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:37,585 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,586 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:-1]], id: ABD8A56FF531456D8D4382DA26C94D4E
2016-05-17 21:52:37,586 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:-1]] ABD8A56FF531456D8D4382DA26C94D4E
2016-05-17 21:52:37,588 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@9988946; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@7d5c5795; comparator=cascading.tuple.hadoop.util.TupleComparator@6d19054; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:37,589 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:37,589 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:37,590 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:37,604 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:37,605 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5c167eac
2016-05-17 21:52:37,607 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5c167eac
2016-05-17 21:52:37,607 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:37,607 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:37,618 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:37,618 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:37,619 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:37,619 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:37,619 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:37,619 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:37,620 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:37,620 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FD7472811FE94572A8A815EDDB0A8842, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:37,622 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,623 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD7472811FE94572A8A815EDDB0A8842, mem on close (mb), free: 158, total: 554, max: 672
2016-05-17 21:52:37,623 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:37,623 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:37,623 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:37,623 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:37,626 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:37 Completed running task attempt: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,630 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:37,631 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521957620, Process_Duration=2, Process_End_Time=1463521957622, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:37,631 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521957355_0001_1_00, eventCount=1
2016-05-17 21:52:37,631 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:37,631 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,632 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521957355_0001_00_000001
2016-05-17 21:52:37,632 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0, startTime=1463521957561, finishTime=1463521957632, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957620, Process_Duration=2, Process_End_Time=1463521957622, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:37,632 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:37,632 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:37,632 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,632 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:37,633 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:37,633 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_FINISHED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, taskId=task_1463521957355_0001_1_00_000000, startTime=1463521957561, finishTime=1463521957633, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1463521957355_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957620, Process_Duration=2, Process_End_Time=1463521957622, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:37,634 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:37,634 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] : 1
2016-05-17 21:52:37,634 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,635 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:37,635 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_FINISHED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, vertexId=vertex_1463521957355_0001_1_00, initRequestedTime=1463521957531, initedTime=1463521957544, startRequestedTime=1463521957545, startedTime=1463521957545, finishTime=1463521957634, timeTaken=89, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957620, Process_Duration=2, Process_End_Time=1463521957622, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521957561, firstTasksToStart=[ task_1463521957355_0001_1_00_000000 ], lastTaskFinishTime=1463521957632, lastTasksToFinish=[ task_1463521957355_0001_1_00_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521957355_0001_1_00_000000 ], longestDurationTasks=[ task_1463521957355_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:37,636 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:37,636 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:37,636 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521957355_0001_00_000001, stoppedTime=1463521957636, exitStatus=0
2016-05-17 21:52:37,636 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:37,636 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:37,636 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] attempt: attempt_1463521957355_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:37,637 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,637 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,637 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:37,637 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521957355_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:37,638 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_STARTED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, taskId=task_1463521957355_0001_1_01_000000, scheduledTime=1463521957638, launchTime=1463521957638
2016-05-17 21:52:37,638 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:37,639 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:37,639 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,639 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown, VertexName: E4FB3BD594E64AE2AAF803F99DE6B0E7, VertexParallelism: 1, TaskAttemptID:attempt_1463521957355_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FD7472811FE94572A8A815EDDB0A8842, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=15C86E44C79B4390A54B7D031375A2B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:37,639 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:37,638 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:37,640 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:37,641 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521957355_0001_00_000002
2016-05-17 21:52:37,641 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:37,643 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521957355_0001_00_000002
2016-05-17 21:52:37,643 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:37,644 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:37,644 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:37,645 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521957355_0001_00_000002, launchTime=1463521957644
2016-05-17 21:52:37,645 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521957355_0001_00_000002
2016-05-17 21:52:37,646 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521957355_0001_1_01_000000_0] to container: [container_1463521957355_0001_00_000002]
2016-05-17 21:52:37,647 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:37,647 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521957355_0001_1_01_000000_0] started. Is using containerId: [container_1463521957355_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:37,648 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0, startTime=1463521957647, containerId=container_1463521957355_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521957355_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:37,648 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:37,648 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:37,648 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:37,649 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521957355_0001_00_000002 given task: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,649 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521957355_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37 Starting to run new task attempt: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,650 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,650 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,650 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:37,650 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:37,650 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521957355_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:37,650 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:37,651 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521957355_0001_00_000001: 19 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:37,651 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521957355_0001_00_000001, Exiting
2016-05-17 21:52:37,651 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521957355_0001_00_000001
2016-05-17 21:52:37,651 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521957355_0001_00_000001
2016-05-17 21:52:37,651 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:37,662 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:37,662 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown, VertexName: E4FB3BD594E64AE2AAF803F99DE6B0E7, VertexParallelism: 1, TaskAttemptID:attempt_1463521957355_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FD7472811FE94572A8A815EDDB0A8842, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=15C86E44C79B4390A54B7D031375A2B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:37,662 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:37,664 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,664 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:37,664 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,665 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521957355_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:37,666 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521957355_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:37,667 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521957355_0001_00_000001 completed successfully
2016-05-17 21:52:37,667 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:37,668 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FD7472811FE94572A8A815EDDB0A8842, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:37,669 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:37,669 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,671 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:37,672 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:37,672 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,673 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=15C86E44C79B4390A54B7D031375A2B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:37,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:37,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 15C86E44C79B4390A54B7D031375A2B9
2016-05-17 21:52:37,674 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:37,674 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 15C86E44C79B4390A54B7D031375A2B9
2016-05-17 21:52:37,676 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:37,690 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:37,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 15C86E44C79B4390A54B7D031375A2B9
2016-05-17 21:52:37,691 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4FB3BD594E64AE2AAF803F99DE6B0E7, ordinal: 1
2016-05-17 21:52:37,691 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4FB3BD594E64AE2AAF803F99DE6B0E7, mem on start (mb), free: 150, total: 554, max: 672
2016-05-17 21:52:37,691 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,691 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:37,692 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:37,692 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:37,692 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:37,692 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:37,693 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:37,693 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:37,693 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:37,694 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:37,694 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FD7472811FE94572A8A815EDDB0A8842, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:37,694 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 15C86E44C79B4390A54B7D031375A2B9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:37,694 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:37,694 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FD7472811FE94572A8A815EDDB0A8842 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:37,695 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,695 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:37,695 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:37,695 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:37,696 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:37,696 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FD7472811FE94572A8A815EDDB0A8842 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:37,696 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:37,696 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:37,697 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:37,697 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,698 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:37,698 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,699 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521957355_0001_1_00_000000_0_10049, runDuration: 57039, ]
2016-05-17 21:52:37,701 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:-1]]
2016-05-17 21:52:37,701 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521957355_0001_1_00_000000_0_10049] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521957701, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 21:52:37,701 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:-1]] streamed: true, id: ABD8A56FF531456D8D4382DA26C94D4E
2016-05-17 21:52:37,702 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"], id: 15C86E44C79B4390A54B7D031375A2B9
2016-05-17 21:52:37,702 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-17 21:52:37,702 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:-1]] ABD8A56FF531456D8D4382DA26C94D4E, for 1 inputs
2016-05-17 21:52:37,701 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,703 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:37,703 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [FD7472811FE94572A8A815EDDB0A8842] #8 in 3ms
2016-05-17 21:52:37,704 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FD7472811FE94572A8A815EDDB0A8842] #8, status:false, isInterrupted:false
2016-05-17 21:52:37,704 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:37,705 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:37,705 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:37,705 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:37,706 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:37,706 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,706 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:37,706 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4FB3BD594E64AE2AAF803F99DE6B0E7, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:52:37,707 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@12b89aea; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@15c28240
2016-05-17 21:52:37,709 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:37,709 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:37,709 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,710 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521957355_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:37,710 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,710 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4FB3BD594E64AE2AAF803F99DE6B0E7, mem on close (mb), free: 147, total: 554, max: 672
2016-05-17 21:52:37,710 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,711 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:37,711 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:37,711 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:37,711 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:37,712 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:37 Completed running task attempt: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,712 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:37,713 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521957706, Process_Duration=4, Process_End_Time=1463521957710, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 21:52:37,713 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:37,713 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,713 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0, startTime=1463521957647, finishTime=1463521957713, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957706, Process_Duration=4, Process_End_Time=1463521957710, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:37,714 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521957355_0001_00_000002
2016-05-17 21:52:37,714 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:37,714 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:37,714 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,714 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:37,715 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:37,715 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_FINISHED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, taskId=task_1463521957355_0001_1_01_000000, startTime=1463521957647, finishTime=1463521957714, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1463521957355_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957706, Process_Duration=4, Process_End_Time=1463521957710, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:37,715 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:37,715 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] : 1
2016-05-17 21:52:37,715 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,716 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:37,717 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_FINISHED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, vertexId=vertex_1463521957355_0001_1_01, initRequestedTime=1463521957533, initedTime=1463521957542, startRequestedTime=1463521957549, startedTime=1463521957549, finishTime=1463521957715, timeTaken=166, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957706, Process_Duration=4, Process_End_Time=1463521957710, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463521957647, firstTasksToStart=[ task_1463521957355_0001_1_01_000000 ], lastTaskFinishTime=1463521957713, lastTasksToFinish=[ task_1463521957355_0001_1_01_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521957355_0001_1_01_000000 ], longestDurationTasks=[ task_1463521957355_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:37,717 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:37,717 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:37,717 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:37,717 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521957355_0001_00_000002, stoppedTime=1463521957717, exitStatus=0
2016-05-17 21:52:37,718 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:37,718 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:37,718 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521957355_0001_1
2016-05-17 21:52:37,719 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521957355_0001_1
2016-05-17 21:52:37,719 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,719 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 15C86E44C79B4390A54B7D031375A2B9 for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,724 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521957355_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:37,724 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521957355_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:37,724 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521957355_0001_00_000002, Exiting
2016-05-17 21:52:37,724 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521957355_0001_00_000002
2016-05-17 21:52:37,725 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521957355_0001_00_000002
2016-05-17 21:52:37,725 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521957355_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:37,725 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521957355_0001_00_000002 completed successfully
2016-05-17 21:52:37,729 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521957355_0001_1, queueSize=0
2016-05-17 21:52:37,729 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:37,730 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521957355_0001_1, startTime=1463521957529, finishTime=1463521957727, timeTaken=198, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=189, FILE_BYTES_WRITTEN=159, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1162870784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043915326, Process_Duration=6, Process_End_Time=2927043915332, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-17 21:52:37,730 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521957355_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:37,730 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521957355_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:37,730 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521957355_0001_00_000002
2016-05-17 21:52:37,730 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:37 Completed Dag: dag_1463521957355_0001_1
2016-05-17 21:52:37,731 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521957355_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:37,731 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521957355_0001_1_post
2016-05-17 21:52:37,731 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,731 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:37,737 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463521957355_0001
2016-05-17 21:52:37,739 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:37,739 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:37,739 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:37,739 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:37,739 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:37,740 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:37,740 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:37,741 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:37,741 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:37,741 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:37,741 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:37,742 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:37,742 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:37,747 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001
2016-05-17 21:52:37,747 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown/_temporary
2016-05-17 21:52:37,748 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.411
2016-05-17 21:52:37,798 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-17 21:52:37,824 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:37,824 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:37,874 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-17 21:52:37,893 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-05-17 21:52:37,894 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:37,894 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:37,894 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:37,918 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-05-17 21:52:37,928 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 21:52:37,930 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:37,930 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:37,938 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-17 21:52:38,076 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-17 21:52:38,076 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:38,077 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:38,077 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-17 21:52:38,078 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  parallel execution of steps is enabled: false
2016-05-17 21:52:38,079 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  executing total steps: 1
2016-05-17 21:52:38,079 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  allocating management threads: 1
2016-05-17 21:52:38,080 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38,080 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:38,099 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-17 21:52:38,105 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:38,105 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:38,105 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:38,117 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001 doesn't exist and is created
2016-05-17 21:52:38,141 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:38,141 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:38,141 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001
2016-05-17 21:52:38,148 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521958105_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:38,165 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:38,167 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:38,169 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:38,170 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:38,170 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001/localmode-log-dir
2016-05-17 21:52:38,171 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001/localmode-log-dir/history.txt.appattempt_1463521958105_0001_000000, maxErrors=10
2016-05-17 21:52:38,171 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:38,172 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521958105_0001_000000, appSubmitTime=1463521958148, launchTime=1463521958148
2016-05-17 21:52:38,176 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:38,178 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:38,179 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41024
2016-05-17 21:52:38,181 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:38,182 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:38,181 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:38,186 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521958105_0001_000000, startTime=1463521958186
2016-05-17 21:52:38,186 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:38,187 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:38,241 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:38,241 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:38,241 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463521958105_0001, dagName=[984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38,259 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38,261 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521958105_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001/localmode-log-dir/dag_1463521958105_0001_1.dot
2016-05-17 21:52:38,262 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001/localmode-log-dir/dag_1463521958105_0001_1-tez-dag.pb.txt
2016-05-17 21:52:38,272 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521958105_0001_1
2016-05-17 21:52:38,272 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,273 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38 Running Dag: dag_1463521958105_0001_1
2016-05-17 21:52:38,286 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:38,287 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521958105_0001_1, submitTime=1463521958260
2016-05-17 21:52:38,287 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,288 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:38,289 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:38,289 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521958105_0001_1, initTime=1463521958287
2016-05-17 21:52:38,289 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521958105_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:38,289 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_STARTED]: dagID=dag_1463521958105_0001_1, startTime=1463521958289
2016-05-17 21:52:38,289 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:38,290 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521958105_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:38,290 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CB9CB66FFDF04BAE947D0404B2C770ED : {76974C0A371B4F67B76EE342D4C1383F={InputName=76974C0A371B4F67B76EE342D4C1383F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:38,290 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 76974C0A371B4F67B76EE342D4C1383F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:38,290 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,290 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,290 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,290 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]. Starting root input initializers: 1
2016-05-17 21:52:38,291 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:38,291 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:38,291 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D761E2268808497B880B1FD23B861F3E : {8B20E3D0427141118675F82D414BEA9E={InputName=8B20E3D0427141118675F82D414BEA9E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:38,291 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8B20E3D0427141118675F82D414BEA9E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:38,293 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 76974C0A371B4F67B76EE342D4C1383F on vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,292 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,295 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,295 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,295 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463521958105_0001, dagName=[984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38,295 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]. Starting root input initializers: 1
2016-05-17 21:52:38,295 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:38,295 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:38,296 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 76974C0A371B4F67B76EE342D4C1383F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:38,296 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8B20E3D0427141118675F82D414BEA9E on vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,296 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,297 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:38,297 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:38,299 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:38,299 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 76974C0A371B4F67B76EE342D4C1383F on vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,300 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:38,300 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,300 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,300 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,300 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=799AFDC84B614A6B903FCE602360466E, vertexId=vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:38,300 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=799AFDC84B614A6B903FCE602360466E, vertexId=vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,302 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:38,302 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8B20E3D0427141118675F82D414BEA9E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:38,302 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=799AFDC84B614A6B903FCE602360466E, vertexName=E9C825134B424E54AB93BFB8D5C11177, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:38,302 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:38,303 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=799AFDC84B614A6B903FCE602360466E, vertexId=vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,303 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:38,304 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:38,305 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:38,305 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, vertexId=vertex_1463521958105_0001_1_02, initRequestedTime=1463521958296, initedTime=1463521958304, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:38,305 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:38,305 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:38,305 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {76974C0A371B4F67B76EE342D4C1383F=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:38,305 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8B20E3D0427141118675F82D414BEA9E on vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,306 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] parallelism set to 1
2016-05-17 21:52:38,306 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:38,306 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521958105_0001_1_01, eventCount=1
2016-05-17 21:52:38,307 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:38,307 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, vertexId=vertex_1463521958105_0001_1_01, initRequestedTime=1463521958290, initedTime=1463521958307, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:38,307 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] with distanceFromRoot: 0
2016-05-17 21:52:38,307 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:38,307 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:38,307 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8B20E3D0427141118675F82D414BEA9E=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:38,308 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] parallelism set to 1
2016-05-17 21:52:38,308 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:38,308 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521958105_0001_1_00, eventCount=1
2016-05-17 21:52:38,308 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:38,309 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D761E2268808497B880B1FD23B861F3E, vertexId=vertex_1463521958105_0001_1_00, initRequestedTime=1463521958291, initedTime=1463521958308, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:38,309 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] with distanceFromRoot: 0
2016-05-17 21:52:38,309 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:38,309 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,309 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,310 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:38,310 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521958105_0001_1_01, startRequestedTime=1463521958309, startedTime=1463521958309
2016-05-17 21:52:38,310 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:38,310 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,310 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,311 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:38,311 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521958105_0001_1_00, startRequestedTime=1463521958310, startedTime=1463521958310
2016-05-17 21:52:38,311 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:38,311 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:38,312 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_STARTED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, taskId=task_1463521958105_0001_1_01_000000, scheduledTime=1463521958311, launchTime=1463521958311
2016-05-17 21:52:38,312 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:38,312 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521958105_0001_1_01 for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] numStartedSources: 1 numSources: 2
2016-05-17 21:52:38,312 INFO  impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] numStartedSources: 1 numSources: 2
2016-05-17 21:52:38,312 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:38,313 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_STARTED]: vertexName=D761E2268808497B880B1FD23B861F3E, taskId=task_1463521958105_0001_1_00_000000, scheduledTime=1463521958312, launchTime=1463521958312
2016-05-17 21:52:38,313 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:38,313 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521958105_0001_1_00 for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] numStartedSources: 2 numSources: 2
2016-05-17 21:52:38,313 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] with distanceFromRoot: 1
2016-05-17 21:52:38,313 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,313 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,314 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CB9CB66FFDF04BAE947D0404B2C770ED in vertex: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,314 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D761E2268808497B880B1FD23B861F3E in vertex: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,314 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E9C825134B424E54AB93BFB8D5C11177 with 2 source tasks and 1 pending tasks
2016-05-17 21:52:38,314 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:38,315 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521958105_0001_1_02, startRequestedTime=1463521958314, startedTime=1463521958314
2016-05-17 21:52:38,315 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:38,315 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: CB9CB66FFDF04BAE947D0404B2C770ED, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=76974C0A371B4F67B76EE342D4C1383F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:38,315 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:38,315 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: D761E2268808497B880B1FD23B861F3E, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8B20E3D0427141118675F82D414BEA9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:38,316 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:38,316 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:38,316 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:38,316 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463521958105_0001, with dag id: dag_1463521958105_0001_1
2016-05-17 21:52:38,317 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:38,317 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:38,318 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:38,318 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521958105_0001_00_000001
2016-05-17 21:52:38,318 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:38,322 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521958105_0001_00_000001
2016-05-17 21:52:38,323 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:38,323 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521958105_0001_00_000002
2016-05-17 21:52:38,323 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:38,324 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521958105_0001_00_000002
2016-05-17 21:52:38,324 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:38,325 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:38,325 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:38,325 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:38,325 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521958105_0001_00_000002, launchTime=1463521958325
2016-05-17 21:52:38,326 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000002
2016-05-17 21:52:38,326 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:38,326 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521958105_0001_00_000001, launchTime=1463521958325
2016-05-17 21:52:38,327 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521958105_0001_1_00_000000_0] to container: [container_1463521958105_0001_00_000002]
2016-05-17 21:52:38,327 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:38,328 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521958105_0001_1_00_000000_0] started. Is using containerId: [container_1463521958105_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:38,328 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000001
2016-05-17 21:52:38,329 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D761E2268808497B880B1FD23B861F3E, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0, startTime=1463521958328, containerId=container_1463521958105_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521958105_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:38,329 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:38,329 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:38,329 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:38,330 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521958105_0001_00_000002 given task: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,330 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521958105_0001_1_01_000000_0] to container: [container_1463521958105_0001_00_000001]
2016-05-17 21:52:38,330 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38 Starting to run new task attempt: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,330 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:38,331 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,331 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,331 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:38,331 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:38,332 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:38,332 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:38,332 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:38,332 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: D761E2268808497B880B1FD23B861F3E, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8B20E3D0427141118675F82D414BEA9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:38,332 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:38,333 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521958105_0001_00_000001 given task: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,333 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,333 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521958105_0001_1_01_000000_0] started. Is using containerId: [container_1463521958105_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:38,334 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,334 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:38 Starting to run new task attempt: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,334 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,334 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,334 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,334 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0, startTime=1463521958333, containerId=container_1463521958105_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521958105_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:38,334 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:38,334 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:38,336 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:38,337 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:38,337 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:38,337 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:38,337 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: CB9CB66FFDF04BAE947D0404B2C770ED, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=76974C0A371B4F67B76EE342D4C1383F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:38,337 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:38,336 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,340 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521958105_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:38,336 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521958105_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:38,336 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:38,341 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:38,338 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,341 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:38,341 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,338 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:38,338 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8B20E3D0427141118675F82D414BEA9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:38,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:38,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:38,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=76974C0A371B4F67B76EE342D4C1383F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:38,345 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:38,346 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,345 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:38,347 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:38,347 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,347 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:38,596 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,596 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,595 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:38,597 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,595 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:38,597 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:38,346 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:38,345 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:38,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,599 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,602 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:38,602 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:38,602 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,602 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,607 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D761E2268808497B880B1FD23B861F3E, ordinal: 1
2016-05-17 21:52:38,607 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D761E2268808497B880B1FD23B861F3E, mem on start (mb), free: 345, total: 370, max: 672
2016-05-17 21:52:38,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:38,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:38,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:38,608 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:38,608 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:38,609 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:38,609 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:38,609 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:38,609 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:38,610 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8B20E3D0427141118675F82D414BEA9E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:38,610 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E9C825134B424E54AB93BFB8D5C11177, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:38,610 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:38,610 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,610 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:38,610 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8B20E3D0427141118675F82D414BEA9E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:38,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:38,611 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:38,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:38,612 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CB9CB66FFDF04BAE947D0404B2C770ED, ordinal: 0
2016-05-17 21:52:38,612 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:38,613 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3251bd19
2016-05-17 21:52:38,612 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,612 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CB9CB66FFDF04BAE947D0404B2C770ED, mem on start (mb), free: 344, total: 370, max: 672
2016-05-17 21:52:38,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:38,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:38,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:38,616 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:38,616 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:38,617 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:38,617 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:38,618 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:38,618 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:38,619 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:38,617 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3251bd19
2016-05-17 21:52:38,619 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:38,619 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:38,619 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,619 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,620 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,619 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E9C825134B424E54AB93BFB8D5C11177, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:38,620 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 76974C0A371B4F67B76EE342D4C1383F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:38,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:38,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 76974C0A371B4F67B76EE342D4C1383F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:38,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:38,621 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:38,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,621 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:38,621 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,621 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:38,622 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,624 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:38,626 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@69ee2790
2016-05-17 21:52:38,625 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@486f9a41; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@16f8468e; comparator=cascading.tuple.hadoop.util.TupleComparator@f93887; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:38,625 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:38,627 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@69ee2790
2016-05-17 21:52:38,628 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:38,628 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:38,627 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:38,629 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:38,629 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:38,628 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,630 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,630 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,634 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@40834bca; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6be4dbdc; comparator=cascading.tuple.hadoop.util.TupleComparator@7e1e7fda; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:38,646 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:38,647 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:38,647 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:38,646 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:38,649 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:38,649 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:38,649 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:38,649 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:38,651 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:38,651 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:38,652 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D761E2268808497B880B1FD23B861F3E, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:38,662 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:38,662 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:38,662 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:38,663 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:38,663 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:38,665 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:38,665 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:38,665 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CB9CB66FFDF04BAE947D0404B2C770ED, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:38,666 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,666 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D761E2268808497B880B1FD23B861F3E, mem on close (mb), free: 141, total: 370, max: 672
2016-05-17 21:52:38,666 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:38,667 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:38,667 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:38,667 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:38,669 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,669 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CB9CB66FFDF04BAE947D0404B2C770ED, mem on close (mb), free: 141, total: 370, max: 672
2016-05-17 21:52:38,669 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:38,670 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:38,670 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:38,670 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:38,671 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:38,673 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:38 Completed running task attempt: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,678 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:38 Completed running task attempt: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,681 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:38,681 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521958666, Process_Duration=3, Process_End_Time=1463521958669, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:38,682 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:38,682 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,682 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000001
2016-05-17 21:52:38,683 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521958105_0001_1_00, eventCount=1
2016-05-17 21:52:38,683 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521958105_0001_1_01, eventCount=1
2016-05-17 21:52:38,684 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:38,684 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D761E2268808497B880B1FD23B861F3E, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0, startTime=1463521958328, finishTime=1463521958683, timeTaken=355, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958652, Process_Duration=14, Process_End_Time=1463521958666, Read_Duration=11, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:38,684 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:38,684 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:38,684 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0, startTime=1463521958333, finishTime=1463521958684, timeTaken=351, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958666, Process_Duration=3, Process_End_Time=1463521958669, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:38,685 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:38,685 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:38,683 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521958652, Process_Duration=14, Process_End_Time=1463521958666, Read_Duration=11, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:38,685 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,686 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:38,687 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_FINISHED]: vertexName=D761E2268808497B880B1FD23B861F3E, taskId=task_1463521958105_0001_1_00_000000, startTime=1463521958328, finishTime=1463521958686, timeTaken=358, status=SUCCEEDED, successfulAttemptID=attempt_1463521958105_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958652, Process_Duration=14, Process_End_Time=1463521958666, Read_Duration=11, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:38,687 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:38,687 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:38,687 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,688 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:38,688 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_FINISHED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, taskId=task_1463521958105_0001_1_01_000000, startTime=1463521958333, finishTime=1463521958687, timeTaken=354, status=SUCCEEDED, successfulAttemptID=attempt_1463521958105_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958666, Process_Duration=3, Process_End_Time=1463521958669, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:38,688 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:38,688 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] : 1
2016-05-17 21:52:38,688 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,689 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:38,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_FINISHED]: vertexName=D761E2268808497B880B1FD23B861F3E, vertexId=vertex_1463521958105_0001_1_00, initRequestedTime=1463521958291, initedTime=1463521958308, startRequestedTime=1463521958310, startedTime=1463521958310, finishTime=1463521958688, timeTaken=378, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958652, Process_Duration=14, Process_End_Time=1463521958666, Read_Duration=11, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521958328, firstTasksToStart=[ task_1463521958105_0001_1_00_000000 ], lastTaskFinishTime=1463521958683, lastTasksToFinish=[ task_1463521958105_0001_1_00_000000 ], minTaskDuration=355, maxTaskDuration=355, avgTaskDuration=355.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521958105_0001_1_00_000000 ], longestDurationTasks=[ task_1463521958105_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:38,690 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:38,690 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:38,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521958105_0001_00_000002, stoppedTime=1463521958690, exitStatus=0
2016-05-17 21:52:38,690 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:38,690 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:38,691 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] : 1
2016-05-17 21:52:38,691 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,691 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:38,692 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_FINISHED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, vertexId=vertex_1463521958105_0001_1_01, initRequestedTime=1463521958290, initedTime=1463521958307, startRequestedTime=1463521958309, startedTime=1463521958309, finishTime=1463521958691, timeTaken=382, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958666, Process_Duration=3, Process_End_Time=1463521958669, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521958333, firstTasksToStart=[ task_1463521958105_0001_1_01_000000 ], lastTaskFinishTime=1463521958684, lastTasksToFinish=[ task_1463521958105_0001_1_01_000000 ], minTaskDuration=351, maxTaskDuration=351, avgTaskDuration=351.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521958105_0001_1_01_000000 ], longestDurationTasks=[ task_1463521958105_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:38,692 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:38,692 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:38,692 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521958105_0001_00_000001, stoppedTime=1463521958692, exitStatus=0
2016-05-17 21:52:38,692 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:38,692 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:38,693 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,692 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:38,693 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000002
2016-05-17 21:52:38,693 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] attempt: attempt_1463521958105_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:38,693 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521958105_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:38,693 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:38,693 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521958105_0001_00_000002, Exiting
2016-05-17 21:52:38,693 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521958105_0001_00_000002
2016-05-17 21:52:38,694 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521958105_0001_00_000002
2016-05-17 21:52:38,693 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:38,699 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521958105_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:38,700 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000001: 18 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:38,700 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521958105_0001_00_000001, Exiting
2016-05-17 21:52:38,700 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521958105_0001_00_000001
2016-05-17 21:52:38,700 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521958105_0001_00_000001
2016-05-17 21:52:38,700 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521958105_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:38,700 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521958105_0001_00_000002 completed successfully
2016-05-17 21:52:38,701 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521958105_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:38,701 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521958105_0001_00_000001 completed successfully
2016-05-17 21:52:38,701 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521958105_0001_00_000002
2016-05-17 21:52:38,702 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] attempt: attempt_1463521958105_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:38,702 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,702 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,702 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:38,703 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521958105_0001_00_000001
2016-05-17 21:52:38,703 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:38,703 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:38,704 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_STARTED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, taskId=task_1463521958105_0001_1_02_000000, scheduledTime=1463521958704, launchTime=1463521958704
2016-05-17 21:52:38,704 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:38,705 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,705 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: E9C825134B424E54AB93BFB8D5C11177, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D761E2268808497B880B1FD23B861F3E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=CB9CB66FFDF04BAE947D0404B2C770ED, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=799AFDC84B614A6B903FCE602360466E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:38,705 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:38,705 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:38,704 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:38,706 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521958105_0001_00_000003
2016-05-17 21:52:38,706 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:38,708 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521958105_0001_00_000003
2016-05-17 21:52:38,708 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:38,710 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:38,711 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521958105_0001_00_000003, launchTime=1463521958710
2016-05-17 21:52:38,710 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:38,711 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000003
2016-05-17 21:52:38,712 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521958105_0001_1_02_000000_0] to container: [container_1463521958105_0001_00_000003]
2016-05-17 21:52:38,713 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:38,713 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521958105_0001_00_000003 given task: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,714 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38 Starting to run new task attempt: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,713 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521958105_0001_1_02_000000_0] started. Is using containerId: [container_1463521958105_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 21:52:38,714 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,715 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0, startTime=1463521958713, containerId=container_1463521958105_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521958105_0001_00_000003/teamcity, completedLogs=
2016-05-17 21:52:38,715 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:38,715 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:38,716 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:38,716 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,717 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:38,717 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:38,717 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:38,717 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:38,718 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:38,718 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: E9C825134B424E54AB93BFB8D5C11177, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D761E2268808497B880B1FD23B861F3E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=CB9CB66FFDF04BAE947D0404B2C770ED, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=799AFDC84B614A6B903FCE602360466E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:38,718 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:38,721 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:38,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,723 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521958105_0001_1_02_000000_0 sent events: (0-2)
2016-05-17 21:52:38,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CB9CB66FFDF04BAE947D0404B2C770ED, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:38,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,725 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:38,725 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,725 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=799AFDC84B614A6B903FCE602360466E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:38,726 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:38,726 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 799AFDC84B614A6B903FCE602360466E
2016-05-17 21:52:38,726 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:38,727 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:38,727 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:38,727 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,727 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,727 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 799AFDC84B614A6B903FCE602360466E
2016-05-17 21:52:38,727 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:38,731 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D761E2268808497B880B1FD23B861F3E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:38,732 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:38,732 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,734 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:38,735 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:38,735 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,736 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,740 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E9C825134B424E54AB93BFB8D5C11177, ordinal: 2
2016-05-17 21:52:38,740 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9C825134B424E54AB93BFB8D5C11177, mem on start (mb), free: 133, total: 370, max: 672
2016-05-17 21:52:38,740 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,740 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-05-17 21:52:38,741 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:38,741 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:38,743 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:38,743 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 799AFDC84B614A6B903FCE602360466E
2016-05-17 21:52:38,743 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:38,743 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:38,744 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:38,744 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:38,744 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:38,744 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:38,745 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-17 21:52:38,745 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-05-17 21:52:38,745 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CB9CB66FFDF04BAE947D0404B2C770ED, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 21:52:38,746 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 799AFDC84B614A6B903FCE602360466E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:38,746 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D761E2268808497B880B1FD23B861F3E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 21:52:38,746 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:38,746 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D761E2268808497B880B1FD23B861F3E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:38,746 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,747 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,747 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CB9CB66FFDF04BAE947D0404B2C770ED being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:38,748 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:38,748 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-05-17 21:52:38,748 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-05-17 21:52:38,748 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:38,749 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:38,749 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:38,749 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D761E2268808497B880B1FD23B861F3E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:38,749 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 21:52:38,749 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CB9CB66FFDF04BAE947D0404B2C770ED with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:38,750 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:38,750 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 21:52:38,750 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:38,750 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:38,751 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:38,751 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,751 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:38,752 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,752 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:38,753 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,753 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521958105_0001_1_00_000000_0_10055, runDuration: 325595, ]
2016-05-17 21:52:38,753 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521958105_0001_1_01_000000_0_10057, runDuration: 331364, ]
2016-05-17 21:52:38,756 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521958105_0001_1_01_000000_0_10057] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521958756, TimeTaken=2, Rate=0.05 MB/s
2016-05-17 21:52:38,756 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,757 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2016-05-17 21:52:38,757 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:38,757 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CB9CB66FFDF04BAE947D0404B2C770ED] #9 in 3ms
2016-05-17 21:52:38,757 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] streamed: true, id: 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,759 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CB9CB66FFDF04BAE947D0404B2C770ED] #9, status:false, isInterrupted:false
2016-05-17 21:52:38,759 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"], id: 799AFDC84B614A6B903FCE602360466E
2016-05-17 21:52:38,760 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:38,760 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-17 21:52:38,760 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 0D56736F1639452F883D131E65A15ADC, for 2 inputs
2016-05-17 21:52:38,760 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 0D56736F1639452F883D131E65A15ADC, for 2 inputs
2016-05-17 21:52:38,761 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:38,761 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:38,761 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:38,762 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:38,762 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,762 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:38,760 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521958105_0001_1_00_000000_0_10055] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521958760, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 21:52:38,765 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,765 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:38,764 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 21:52:38,766 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 21:52:38,766 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D761E2268808497B880B1FD23B861F3E] #10 in 7ms
2016-05-17 21:52:38,766 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D761E2268808497B880B1FD23B861F3E] #10, status:false, isInterrupted:false
2016-05-17 21:52:39,046 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:39,046 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:39,047 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:39,048 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:39,048 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:39,049 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:39,049 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:39,049 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:39,049 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:39,049 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E9C825134B424E54AB93BFB8D5C11177, all 2 inputs ready in: 00:00:00.283
2016-05-17 21:52:39,051 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@30bad9de; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@3b3596d0
2016-05-17 21:52:39,054 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@3b40500e; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7b47e585
2016-05-17 21:52:39,057 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:39,057 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:39,058 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39,058 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521958105_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 21:52:39,059 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39,060 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9C825134B424E54AB93BFB8D5C11177, mem on close (mb), free: 519, total: 543, max: 672
2016-05-17 21:52:39,060 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:39,060 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:39,062 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:39,062 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:39,062 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:39,062 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:39,062 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:39,063 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:39,064 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:39,064 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:39,064 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:39,064 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:39 Completed running task attempt: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39,065 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:39,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=280, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521959050, Process_Duration=9, Process_End_Time=1463521959059, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 21:52:39,067 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0, startTime=1463521958713, finishTime=1463521959066, timeTaken=353, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=280, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521959050, Process_Duration=9, Process_End_Time=1463521959059, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:39,067 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:39,067 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:39,068 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,068 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:39,069 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000003
2016-05-17 21:52:39,069 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39,069 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:39,069 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_FINISHED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, taskId=task_1463521958105_0001_1_02_000000, startTime=1463521958713, finishTime=1463521959069, timeTaken=356, status=SUCCEEDED, successfulAttemptID=attempt_1463521958105_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=280, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521959050, Process_Duration=9, Process_End_Time=1463521959059, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:39,069 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:39,070 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] : 1
2016-05-17 21:52:39,070 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:39,071 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_FINISHED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, vertexId=vertex_1463521958105_0001_1_02, initRequestedTime=1463521958296, initedTime=1463521958304, startRequestedTime=1463521958314, startedTime=1463521958314, finishTime=1463521959070, timeTaken=756, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=280, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521959050, Process_Duration=9, Process_End_Time=1463521959059, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463521958713, firstTasksToStart=[ task_1463521958105_0001_1_02_000000 ], lastTaskFinishTime=1463521959066, lastTasksToFinish=[ task_1463521958105_0001_1_02_000000 ], minTaskDuration=353, maxTaskDuration=353, avgTaskDuration=353.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521958105_0001_1_02_000000 ], longestDurationTasks=[ task_1463521958105_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:39,071 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:39,071 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:39,071 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521958105_0001_00_000003, stoppedTime=1463521959071, exitStatus=0
2016-05-17 21:52:39,071 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:39,071 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:39,071 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:39,072 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521958105_0001_1
2016-05-17 21:52:39,072 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521958105_0001_1
2016-05-17 21:52:39,072 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 799AFDC84B614A6B903FCE602360466E for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:39,072 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:39,073 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:39,079 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521958105_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 21:52:39,079 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:39,080 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521958105_0001_00_000003, Exiting
2016-05-17 21:52:39,080 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521958105_0001_00_000003
2016-05-17 21:52:39,080 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521958105_0001_00_000003
2016-05-17 21:52:39,080 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521958105_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:39,081 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521958105_0001_00_000003 completed successfully
2016-05-17 21:52:39,081 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:39,081 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:39,084 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521958105_0001_1, queueSize=0
2016-05-17 21:52:39,084 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:39,084 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521958105_0001_1, startTime=1463521958289, finishTime=1463521959081, timeTaken=792, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=470, FILE_BYTES_WRITTEN=1600, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=770, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1346895872, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390565876368, Process_Duration=26, Process_End_Time=4390565876394, Read_Duration=13, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-17 21:52:39,084 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521958105_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:39,084 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521958105_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:39,085 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521958105_0001_00_000003
2016-05-17 21:52:39,085 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:39 Completed Dag: dag_1463521958105_0001_1
2016-05-17 21:52:39,086 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521958105_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:39,086 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521958105_0001_1_post
2016-05-17 21:52:39,086 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,086 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:39,093 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463521958105_0001
2016-05-17 21:52:39,096 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:39,096 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:39,096 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:39,096 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:39,096 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:39,097 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:39,097 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:39,097 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:39,097 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:39,097 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:39,098 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:39,098 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:39,099 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:39,104 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001
2016-05-17 21:52:39,104 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup/_temporary
2016-05-17 21:52:39,105 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  completed in: 00:01.028
2016-05-17 21:52:39,190 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-17 21:52:39,213 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:39,213 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:39,250 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:52:39,259 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 21:52:39,259 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:39,259 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:39,260 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:39,276 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:39,277 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-17 21:52:39,285 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:39,285 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:39,285 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-17 21:52:39,286 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:39,286 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:39,286 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:39,286 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...tform/testReplace/replace
2016-05-17 21:52:39,287 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:39,298 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:39,302 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:39,303 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:39,303 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:39,312 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001 doesn't exist and is created
2016-05-17 21:52:39,328 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:39,328 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:39,329 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001
2016-05-17 21:52:39,332 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521959303_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:39,343 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:39,344 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:39,345 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:39,345 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:39,346 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001/localmode-log-dir
2016-05-17 21:52:39,346 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001/localmode-log-dir/history.txt.appattempt_1463521959303_0001_000000, maxErrors=10
2016-05-17 21:52:39,346 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:39,346 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521959303_0001_000000, appSubmitTime=1463521959332, launchTime=1463521959332
2016-05-17 21:52:39,350 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:39,351 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:39,352 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:39,352 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:39,357 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:32975
2016-05-17 21:52:39,358 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:39,360 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521959303_0001_000000, startTime=1463521959360
2016-05-17 21:52:39,360 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:39,360 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:39,429 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:39,429 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:39,429 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463521959303_0001, dagName=[65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace
2016-05-17 21:52:39,452 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace
2016-05-17 21:52:39,454 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521959303_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001/localmode-log-dir/dag_1463521959303_0001_1.dot
2016-05-17 21:52:39,459 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001/localmode-log-dir/dag_1463521959303_0001_1-tez-dag.pb.txt
2016-05-17 21:52:39,465 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521959303_0001_1
2016-05-17 21:52:39,465 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,465 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace
2016-05-17 21:52:39 Running Dag: dag_1463521959303_0001_1
2016-05-17 21:52:39,477 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521959303_0001_1, submitTime=1463521959453
2016-05-17 21:52:39,477 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:39,478 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C8389DC629D94E66B1956D1FFCAFA945
2016-05-17 21:52:39,478 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:39,479 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521959303_0001_1, initTime=1463521959477
2016-05-17 21:52:39,479 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:39,479 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959303_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:39,479 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_STARTED]: dagID=dag_1463521959303_0001_1, startTime=1463521959479
2016-05-17 21:52:39,480 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:39,480 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:39,480 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959303_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:39,481 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C8389DC629D94E66B1956D1FFCAFA945 : {85AA824A7BA14D46A42C4E1E5B430590={InputName=85AA824A7BA14D46A42C4E1E5B430590}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:39,481 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 85AA824A7BA14D46A42C4E1E5B430590, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:39,481 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,481 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463521959303_0001, dagName=[65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace
2016-05-17 21:52:39,481 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,482 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,483 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]. Starting root input initializers: 1
2016-05-17 21:52:39,485 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:39,485 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 85AA824A7BA14D46A42C4E1E5B430590 on vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,487 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:39,488 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 85AA824A7BA14D46A42C4E1E5B430590 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:39,488 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:39,488 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:39,491 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:39,491 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 85AA824A7BA14D46A42C4E1E5B430590 on vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,492 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:39,492 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {85AA824A7BA14D46A42C4E1E5B430590=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:39,492 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] parallelism set to 1
2016-05-17 21:52:39,492 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:39,492 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521959303_0001_1_00, eventCount=1
2016-05-17 21:52:39,493 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,493 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2B86812B7FBC4CD59E4441D15694232E, vertexId=vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:39,493 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2B86812B7FBC4CD59E4441D15694232E, vertexId=vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,495 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2B86812B7FBC4CD59E4441D15694232E, vertexName=C8389DC629D94E66B1956D1FFCAFA945, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:39,495 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2B86812B7FBC4CD59E4441D15694232E, vertexId=vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,496 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:39,497 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, vertexId=vertex_1463521959303_0001_1_00, initRequestedTime=1463521959481, initedTime=1463521959496, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:39,497 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] with distanceFromRoot: 0
2016-05-17 21:52:39,497 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:39,497 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C8389DC629D94E66B1956D1FFCAFA945
2016-05-17 21:52:39,497 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,498 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:39,498 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521959303_0001_1_00, startRequestedTime=1463521959497, startedTime=1463521959497
2016-05-17 21:52:39,498 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:39,500 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:39,503 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:TASK_STARTED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, taskId=task_1463521959303_0001_1_00_000000, scheduledTime=1463521959498, launchTime=1463521959498
2016-05-17 21:52:39,503 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959303_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:39,503 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,505 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace, VertexName: C8389DC629D94E66B1956D1FFCAFA945, VertexParallelism: 1, TaskAttemptID:attempt_1463521959303_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=85AA824A7BA14D46A42C4E1E5B430590, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2B86812B7FBC4CD59E4441D15694232E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:39,505 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959303_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:39,505 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:39,506 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:39,506 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:39,507 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:39,507 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521959303_0001_00_000001
2016-05-17 21:52:39,507 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:39,509 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521959303_0001, with dag id: dag_1463521959303_0001_1
2016-05-17 21:52:39,509 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521959303_0001_00_000001
2016-05-17 21:52:39,510 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:39,510 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:39,511 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:39,511 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521959303_0001_00_000001, launchTime=1463521959510
2016-05-17 21:52:39,512 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959303_0001_00_000001
2016-05-17 21:52:39,514 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521959303_0001_1_00_000000_0] to container: [container_1463521959303_0001_00_000001]
2016-05-17 21:52:39,514 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:39,514 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521959303_0001_1_00_000000_0] started. Is using containerId: [container_1463521959303_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:39,515 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0, startTime=1463521959514, containerId=container_1463521959303_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521959303_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:39,515 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959303_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:39,515 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959303_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:39,515 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:39,517 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521959303_0001_00_000001 given task: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,517 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959303_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39 Starting to run new task attempt: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,517 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,518 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,518 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:39,518 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:39,518 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:39,518 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:39,519 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:39,519 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace, VertexName: C8389DC629D94E66B1956D1FFCAFA945, VertexParallelism: 1, TaskAttemptID:attempt_1463521959303_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=85AA824A7BA14D46A42C4E1E5B430590, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2B86812B7FBC4CD59E4441D15694232E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:39,519 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:39,520 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:39,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,522 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2B86812B7FBC4CD59E4441D15694232E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:39,522 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:39,522 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2B86812B7FBC4CD59E4441D15694232E
2016-05-17 21:52:39,523 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:39,523 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2B86812B7FBC4CD59E4441D15694232E
2016-05-17 21:52:39,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=85AA824A7BA14D46A42C4E1E5B430590, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:39,523 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521959303_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:39,524 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:39,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:39,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,526 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,531 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:39,532 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,532 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,534 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:39,534 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2B86812B7FBC4CD59E4441D15694232E
2016-05-17 21:52:39,535 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8389DC629D94E66B1956D1FFCAFA945, ordinal: 0
2016-05-17 21:52:39,536 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8389DC629D94E66B1956D1FFCAFA945, mem on start (mb), free: 442, total: 543, max: 672
2016-05-17 21:52:39,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:39,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:39,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:39,536 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:39,536 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:39,538 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:39,538 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:39,538 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:39,538 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:39,539 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:39,539 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2B86812B7FBC4CD59E4441D15694232E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:39,539 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 85AA824A7BA14D46A42C4E1E5B430590, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:39,539 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:39,539 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,539 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:39,539 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,539 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 85AA824A7BA14D46A42C4E1E5B430590 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:39,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:39,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:39,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:39,541 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,542 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:39,542 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@65bfce1d
2016-05-17 21:52:39,543 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@65bfce1d
2016-05-17 21:52:39,543 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:39,543 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:39,544 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:39,544 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,545 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"], id: 2B86812B7FBC4CD59E4441D15694232E
2016-05-17 21:52:39,545 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-17 21:52:39,545 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:39,545 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:39,545 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8389DC629D94E66B1956D1FFCAFA945, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:39,547 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:39,547 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:39,548 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,548 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521959303_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:39,549 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,549 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8389DC629D94E66B1956D1FFCAFA945, mem on close (mb), free: 440, total: 543, max: 672
2016-05-17 21:52:39 Completed running task attempt: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,549 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:39,550 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-17 21:52:39,550 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:39,550 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:39,550 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,551 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959303_0001_00_000001
2016-05-17 21:52:39,550 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0, startTime=1463521959514, finishTime=1463521959550, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:39,551 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959303_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:39,551 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:39,551 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,553 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:39,553 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:TASK_FINISHED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, taskId=task_1463521959303_0001_1_00_000000, startTime=1463521959514, finishTime=1463521959553, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463521959303_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:39,554 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959303_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:39,554 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] : 1
2016-05-17 21:52:39,554 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,556 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:VERTEX_FINISHED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, vertexId=vertex_1463521959303_0001_1_00, initRequestedTime=1463521959481, initedTime=1463521959496, startRequestedTime=1463521959497, startedTime=1463521959497, finishTime=1463521959554, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463521959514, firstTasksToStart=[ task_1463521959303_0001_1_00_000000 ], lastTaskFinishTime=1463521959550, lastTasksToFinish=[ task_1463521959303_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521959303_0001_1_00_000000 ], longestDurationTasks=[ task_1463521959303_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:39,556 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:39,556 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:39,556 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521959303_0001_00_000001, stoppedTime=1463521959556, exitStatus=0
2016-05-17 21:52:39,556 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:39,556 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:39,556 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521959303_0001_1
2016-05-17 21:52:39,557 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521959303_0001_1
2016-05-17 21:52:39,557 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2B86812B7FBC4CD59E4441D15694232E for vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,559 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:39,559 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:39,561 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521959303_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:39,561 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959303_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:39,561 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521959303_0001_00_000001, Exiting
2016-05-17 21:52:39,561 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521959303_0001_00_000001
2016-05-17 21:52:39,562 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521959303_0001_00_000001
2016-05-17 21:52:39,563 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521959303_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:39,564 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521959303_0001_00_000001 completed successfully
2016-05-17 21:52:39,568 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521959303_0001_1, queueSize=0
2016-05-17 21:52:39,569 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:39,569 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521959303_0001_1, startTime=1463521959479, finishTime=1463521959567, timeTaken=88, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:39,569 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521959303_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:39,570 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959303_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:39,570 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:39,570 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521959303_0001_00_000001
2016-05-17 21:52:39 Completed Dag: dag_1463521959303_0001_1
2016-05-17 21:52:39,571 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521959303_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:39,571 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521959303_0001_1_post
2016-05-17 21:52:39,571 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,571 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:39,577 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463521959303_0001
2016-05-17 21:52:39,579 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:39,579 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:39,579 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:39,580 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:39,580 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:39,580 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:39,581 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:39,581 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:39,581 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:39,582 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:39,582 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:39,582 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:39,582 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:39,586 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001
2016-05-17 21:52:39,586 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace/_temporary
2016-05-17 21:52:39,587 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.301
2016-05-17 21:52:39,621 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-17 21:52:39,643 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:39,643 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:39,678 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:52:39,689 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 21:52:39,689 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:39,689 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:39,690 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:39,704 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 21:52:39,710 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-17 21:52:39,756 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 21:52:39,757 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:39,757 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-17 21:52:39,757 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  parallel execution of steps is enabled: false
2016-05-17 21:52:39,757 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  executing total steps: 1
2016-05-17 21:52:39,757 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  allocating management threads: 1
2016-05-17 21:52:39,758 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount2/count2
2016-05-17 21:52:39,758 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:39,770 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 21:52:39,774 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:39,775 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:39,775 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:39,785 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001 doesn't exist and is created
2016-05-17 21:52:39,803 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:39,804 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:39,804 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001
2016-05-17 21:52:39,807 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521959775_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:39,817 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:39,817 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:39,819 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:39,819 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:39,819 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001/localmode-log-dir
2016-05-17 21:52:39,820 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001/localmode-log-dir/history.txt.appattempt_1463521959775_0001_000000, maxErrors=10
2016-05-17 21:52:39,820 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:39,820 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521959775_0001_000000, appSubmitTime=1463521959806, launchTime=1463521959807
2016-05-17 21:52:39,824 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:39,826 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:39,827 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:39,827 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:39,831 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54977
2016-05-17 21:52:39,833 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:39,835 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521959775_0001_000000, startTime=1463521959835
2016-05-17 21:52:39,835 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:39,835 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:39,904 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:39,904 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:39,904 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463521959775_0001, dagName=[D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2
2016-05-17 21:52:39,920 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2
2016-05-17 21:52:39,921 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521959775_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001/localmode-log-dir/dag_1463521959775_0001_1.dot
2016-05-17 21:52:39,922 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001/localmode-log-dir/dag_1463521959775_0001_1-tez-dag.pb.txt
2016-05-17 21:52:39,932 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521959775_0001_1
2016-05-17 21:52:39,932 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,932 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2
2016-05-17 21:52:39 Running Dag: dag_1463521959775_0001_1
2016-05-17 21:52:39,944 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:39,945 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521959775_0001_1, submitTime=1463521959920
2016-05-17 21:52:39,946 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,946 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:39,946 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:39,947 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521959775_0001_1, initTime=1463521959945
2016-05-17 21:52:39,948 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959775_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:39,948 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:39,948 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_STARTED]: dagID=dag_1463521959775_0001_1, startTime=1463521959948
2016-05-17 21:52:39,948 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:39,949 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463521959775_0001, dagName=[D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2
2016-05-17 21:52:39,949 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959775_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:39,949 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E50C8771203D4054B9211097074879AF : {69B3D97299504777947459B764F6AA36={InputName=69B3D97299504777947459B764F6AA36}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:39,949 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 69B3D97299504777947459B764F6AA36, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:39,949 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,949 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,949 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,950 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]. Starting root input initializers: 1
2016-05-17 21:52:39,950 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:39,950 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,953 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:39,953 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,953 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,953 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,953 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0FE2DE5D6F974A9CBA974E77810B04FD, vertexId=vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:39,953 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0FE2DE5D6F974A9CBA974E77810B04FD, vertexId=vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,954 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 69B3D97299504777947459B764F6AA36 on vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,955 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0FE2DE5D6F974A9CBA974E77810B04FD, vertexName=DB8E2F2045794DDEAF203EE1705A8C16, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:39,955 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0FE2DE5D6F974A9CBA974E77810B04FD, vertexId=vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,957 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:39,957 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:39,957 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, vertexId=vertex_1463521959775_0001_1_01, initRequestedTime=1463521959950, initedTime=1463521959957, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:39,957 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:39,957 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 69B3D97299504777947459B764F6AA36 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:39,958 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:39,959 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:39,960 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 21:52:39,960 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 69B3D97299504777947459B764F6AA36 on vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,960 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:39,961 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {69B3D97299504777947459B764F6AA36=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:39,961 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] parallelism set to 1
2016-05-17 21:52:39,961 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:39,962 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521959775_0001_1_00, eventCount=1
2016-05-17 21:52:39,962 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:39,962 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E50C8771203D4054B9211097074879AF, vertexId=vertex_1463521959775_0001_1_00, initRequestedTime=1463521959949, initedTime=1463521959962, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:39,962 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] with distanceFromRoot: 0
2016-05-17 21:52:39,963 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:39,963 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E50C8771203D4054B9211097074879AF
2016-05-17 21:52:39,963 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,963 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:39,963 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521959775_0001_1_00, startRequestedTime=1463521959963, startedTime=1463521959963
2016-05-17 21:52:39,964 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:39,964 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:39,964 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_STARTED]: vertexName=E50C8771203D4054B9211097074879AF, taskId=task_1463521959775_0001_1_00_000000, scheduledTime=1463521959964, launchTime=1463521959964
2016-05-17 21:52:39,965 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:39,965 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521959775_0001_1_00 for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] numStartedSources: 1 numSources: 1
2016-05-17 21:52:39,965 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] with distanceFromRoot: 1
2016-05-17 21:52:39,965 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,966 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E50C8771203D4054B9211097074879AF in vertex: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,966 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DB8E2F2045794DDEAF203EE1705A8C16 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:39,966 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:39,966 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521959775_0001_1_01, startRequestedTime=1463521959966, startedTime=1463521959966
2016-05-17 21:52:39,967 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:39,967 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2, VertexName: E50C8771203D4054B9211097074879AF, VertexParallelism: 1, TaskAttemptID:attempt_1463521959775_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=69B3D97299504777947459B764F6AA36, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DB8E2F2045794DDEAF203EE1705A8C16, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:39,967 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463521959775_0001, with dag id: dag_1463521959775_0001_1
2016-05-17 21:52:39,967 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:39,968 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:39,969 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:39,969 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:39,970 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:39,971 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521959775_0001_00_000001
2016-05-17 21:52:39,971 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:39,972 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521959775_0001_00_000001
2016-05-17 21:52:39,972 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:39,973 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:39,974 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:39,974 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521959775_0001_00_000001, launchTime=1463521959973
2016-05-17 21:52:39,975 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959775_0001_00_000001
2016-05-17 21:52:39,976 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521959775_0001_1_00_000000_0] to container: [container_1463521959775_0001_00_000001]
2016-05-17 21:52:39,976 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:39,977 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521959775_0001_00_000001 given task: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39,977 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959775_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39 Starting to run new task attempt: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39,978 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39,978 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,978 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:39,978 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521959775_0001_1_00_000000_0] started. Is using containerId: [container_1463521959775_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:39,978 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:39,979 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:39,979 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:39,979 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:39,979 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2, VertexName: E50C8771203D4054B9211097074879AF, VertexParallelism: 1, TaskAttemptID:attempt_1463521959775_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=69B3D97299504777947459B764F6AA36, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DB8E2F2045794DDEAF203EE1705A8C16, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:39,979 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:39,980 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:39,980 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E50C8771203D4054B9211097074879AF, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0, startTime=1463521959978, containerId=container_1463521959775_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521959775_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:39,980 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:39,980 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:39,983 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521959775_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:39,983 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39,984 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:39,984 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,985 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=69B3D97299504777947459B764F6AA36, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:39,986 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:39,987 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:39,989 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:39,988 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,989 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DB8E2F2045794DDEAF203EE1705A8C16, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:39,991 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:39,991 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,992 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:39,992 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:39,992 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,997 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:39,997 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:39,997 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:39,998 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E50C8771203D4054B9211097074879AF, ordinal: 0
2016-05-17 21:52:39,998 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E50C8771203D4054B9211097074879AF, mem on start (mb), free: 358, total: 543, max: 672
2016-05-17 21:52:39,998 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,999 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:39,999 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:39,999 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:39,999 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:39,999 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:39,999 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:39,999 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:40,000 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:40,000 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:40,000 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 69B3D97299504777947459B764F6AA36, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:40,001 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DB8E2F2045794DDEAF203EE1705A8C16, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:40,001 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:40,001 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 69B3D97299504777947459B764F6AA36 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:40,001 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:40,002 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:40,002 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:40,002 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:40,003 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:40,003 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:40,003 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40,003 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:40,004 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@e59e47d
2016-05-17 21:52:40,005 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@e59e47d
2016-05-17 21:52:40,005 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:40,005 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:40,006 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,007 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:40,007 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: E98B27387D99441FA4FC8D844CEF0506
2016-05-17 21:52:40,007 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] E98B27387D99441FA4FC8D844CEF0506
2016-05-17 21:52:40,009 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2ce0692f; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@4119a3ad; comparator=cascading.tuple.hadoop.util.TupleComparator@7a48d43e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:40,009 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:40,009 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:40,010 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:40,029 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:40,030 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:40,030 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:40,030 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:40,031 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:40,031 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,031 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:40,032 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E50C8771203D4054B9211097074879AF, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:40,034 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40,034 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E50C8771203D4054B9211097074879AF, mem on close (mb), free: 425, total: 556, max: 672
2016-05-17 21:52:40,034 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:40,035 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:40,035 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 21:52:40,035 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:52:40,039 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40,043 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:40,044 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521959775_0001_1_00, eventCount=1
2016-05-17 21:52:40,045 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:40,045 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E50C8771203D4054B9211097074879AF, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0, startTime=1463521959978, finishTime=1463521960044, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960032, Process_Duration=2, Process_End_Time=1463521960034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:40,045 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:40,045 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40,046 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_FINISHED]: vertexName=E50C8771203D4054B9211097074879AF, taskId=task_1463521959775_0001_1_00_000000, startTime=1463521959978, finishTime=1463521960045, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1463521959775_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960032, Process_Duration=2, Process_End_Time=1463521960034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:40,046 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:40,046 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] : 1
2016-05-17 21:52:40,046 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:40,047 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:40,048 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:40,048 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_FINISHED]: vertexName=E50C8771203D4054B9211097074879AF, vertexId=vertex_1463521959775_0001_1_00, initRequestedTime=1463521959949, initedTime=1463521959962, startRequestedTime=1463521959963, startedTime=1463521959963, finishTime=1463521960047, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960032, Process_Duration=2, Process_End_Time=1463521960034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463521959978, firstTasksToStart=[ task_1463521959775_0001_1_00_000000 ], lastTaskFinishTime=1463521960044, lastTasksToFinish=[ task_1463521959775_0001_1_00_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521959775_0001_1_00_000000 ], longestDurationTasks=[ task_1463521959775_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:40,048 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:40,049 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] attempt: attempt_1463521959775_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:40,049 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:40,049 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:40,050 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:40,049 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:40,051 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_STARTED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, taskId=task_1463521959775_0001_1_01_000000, scheduledTime=1463521960050, launchTime=1463521960050
2016-05-17 21:52:40,051 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:40,051 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:40,051 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:40,052 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:40,052 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521959775_0001_00_000001, stoppedTime=1463521960052, exitStatus=0
2016-05-17 21:52:40,052 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:40,052 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:40,053 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2, VertexName: DB8E2F2045794DDEAF203EE1705A8C16, VertexParallelism: 1, TaskAttemptID:attempt_1463521959775_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E50C8771203D4054B9211097074879AF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0FE2DE5D6F974A9CBA974E77810B04FD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:40,053 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:40,053 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:40,053 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521959775_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:40,054 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:40,054 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521959775_0001_00_000002
2016-05-17 21:52:40,054 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:40,053 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521960032, Process_Duration=2, Process_End_Time=1463521960034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:52:40,055 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:40,055 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,055 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959775_0001_00_000001
2016-05-17 21:52:40,056 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521959775_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:40,056 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959775_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:40,056 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521959775_0001_00_000001, Exiting
2016-05-17 21:52:40,056 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521959775_0001_00_000001
2016-05-17 21:52:40,057 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521959775_0001_00_000001
2016-05-17 21:52:40,056 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521959775_0001_00_000002
2016-05-17 21:52:40,060 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:40,060 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521959775_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:40,061 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521959775_0001_00_000001 completed successfully
2016-05-17 21:52:40,061 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:40,065 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:40,065 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:40,065 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521959775_0001_00_000002, launchTime=1463521960065
2016-05-17 21:52:40,066 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959775_0001_00_000002
2016-05-17 21:52:40,066 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521959775_0001_1_01_000000_0] to container: [container_1463521959775_0001_00_000002]
2016-05-17 21:52:40,066 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:40,067 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521959775_0001_1_01_000000_0] started. Is using containerId: [container_1463521959775_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:40,067 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0, startTime=1463521960067, containerId=container_1463521959775_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521959775_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:40,067 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:40,067 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:40,067 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:40,067 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521959775_0001_00_000002 given task: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,068 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959775_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40 Starting to run new task attempt: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,068 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,068 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,068 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:40,068 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:40,069 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:40,069 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:40,069 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:40,069 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2, VertexName: DB8E2F2045794DDEAF203EE1705A8C16, VertexParallelism: 1, TaskAttemptID:attempt_1463521959775_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E50C8771203D4054B9211097074879AF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0FE2DE5D6F974A9CBA974E77810B04FD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:40,069 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:40,070 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:40,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,071 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E50C8771203D4054B9211097074879AF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:40,071 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:40,071 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,072 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:40,073 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:40,073 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,074 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521959775_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:40,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0FE2DE5D6F974A9CBA974E77810B04FD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:40,076 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:40,077 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0FE2DE5D6F974A9CBA974E77810B04FD
2016-05-17 21:52:40,077 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:40,077 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:40,077 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0FE2DE5D6F974A9CBA974E77810B04FD
2016-05-17 21:52:40,084 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DB8E2F2045794DDEAF203EE1705A8C16, ordinal: 1
2016-05-17 21:52:40,084 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DB8E2F2045794DDEAF203EE1705A8C16, mem on start (mb), free: 419, total: 556, max: 672
2016-05-17 21:52:40,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:40,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:40,087 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:40,087 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0FE2DE5D6F974A9CBA974E77810B04FD
2016-05-17 21:52:40,088 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:40,088 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:40,088 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:40,089 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:40,090 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:40,090 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:40,092 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:40,092 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E50C8771203D4054B9211097074879AF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:40,092 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0FE2DE5D6F974A9CBA974E77810B04FD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:40,093 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:40,093 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,094 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:40,095 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:40,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E50C8771203D4054B9211097074879AF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:40,095 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E50C8771203D4054B9211097074879AF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:40,095 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:40,095 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:40,095 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:40,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:40,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:40,097 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:40,098 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,101 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521959775_0001_1_00_000000_0_10068, runDuration: 50942, ]
2016-05-17 21:52:40,102 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 21:52:40,103 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521959775_0001_1_00_000000_0_10068] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463521960103, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:52:40,103 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,103 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:40,103 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: E98B27387D99441FA4FC8D844CEF0506
2016-05-17 21:52:40,104 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E50C8771203D4054B9211097074879AF] #11 in 1ms
2016-05-17 21:52:40,104 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"], id: 0FE2DE5D6F974A9CBA974E77810B04FD
2016-05-17 21:52:40,104 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E50C8771203D4054B9211097074879AF] #11, status:false, isInterrupted:false
2016-05-17 21:52:40,105 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:40,105 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-17 21:52:40,106 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] E98B27387D99441FA4FC8D844CEF0506, for 1 inputs
2016-05-17 21:52:40,106 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 21:52:40,106 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:40,107 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:40,107 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 21:52:40,107 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,107 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:40,107 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DB8E2F2045794DDEAF203EE1705A8C16, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:52:40,109 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@177dd5b4; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@46592878
2016-05-17 21:52:40,113 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:40,114 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:40,114 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:40,114 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:40,114 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,115 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521959775_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:40,115 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,115 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DB8E2F2045794DDEAF203EE1705A8C16, mem on close (mb), free: 416, total: 556, max: 672
2016-05-17 21:52:40,116 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,116 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:40,117 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:40,117 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:40,117 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:40,117 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,118 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:40,119 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:40,119 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521960107, Process_Duration=8, Process_End_Time=1463521960115, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 21:52:40,120 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0, startTime=1463521960067, finishTime=1463521960119, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960107, Process_Duration=8, Process_End_Time=1463521960115, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:40,120 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:40,120 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:40,121 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:40,121 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,120 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,121 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959775_0001_00_000002
2016-05-17 21:52:40,122 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_FINISHED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, taskId=task_1463521959775_0001_1_01_000000, startTime=1463521960067, finishTime=1463521960121, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463521959775_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960107, Process_Duration=8, Process_End_Time=1463521960115, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:40,122 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:40,122 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] : 1
2016-05-17 21:52:40,122 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:40,122 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:40,123 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:40,123 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_FINISHED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, vertexId=vertex_1463521959775_0001_1_01, initRequestedTime=1463521959950, initedTime=1463521959957, startRequestedTime=1463521959966, startedTime=1463521959966, finishTime=1463521960122, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960107, Process_Duration=8, Process_End_Time=1463521960115, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463521960067, firstTasksToStart=[ task_1463521959775_0001_1_01_000000 ], lastTaskFinishTime=1463521960119, lastTasksToFinish=[ task_1463521959775_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521959775_0001_1_01_000000 ], longestDurationTasks=[ task_1463521959775_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:40,124 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:40,124 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:40,124 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521959775_0001_00_000002, stoppedTime=1463521960124, exitStatus=0
2016-05-17 21:52:40,124 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:40,124 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:40,124 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521959775_0001_1
2016-05-17 21:52:40,125 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521959775_0001_1
2016-05-17 21:52:40,125 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:40,125 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0FE2DE5D6F974A9CBA974E77810B04FD for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:40,124 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:40,131 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521959775_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:40,132 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959775_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:40,132 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521959775_0001_00_000002, Exiting
2016-05-17 21:52:40,132 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521959775_0001_00_000002
2016-05-17 21:52:40,132 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521959775_0001_00_000002
2016-05-17 21:52:40,133 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521959775_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:40,133 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521959775_0001_00_000002 completed successfully
2016-05-17 21:52:40,135 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521959775_0001_1, queueSize=0
2016-05-17 21:52:40,135 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:40,136 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521959775_0001_1, startTime=1463521959948, finishTime=1463521960133, timeTaken=185, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1167065088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043920139, Process_Duration=10, Process_End_Time=2927043920149, Read_Duration=3, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 21:52:40,137 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521959775_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:40,137 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959775_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:40,138 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:40,138 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521959775_0001_00_000002
2016-05-17 21:52:40 Completed Dag: dag_1463521959775_0001_1
2016-05-17 21:52:40,139 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521959775_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:40,140 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521959775_0001_1_post
2016-05-17 21:52:40,140 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,140 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:40,145 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463521959775_0001
2016-05-17 21:52:40,148 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:40,148 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:40,148 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:40,149 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:40,149 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:40,149 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:40,150 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:40,150 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:40,150 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:40,151 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:40,151 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:40,151 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:40,152 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:40,157 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001
2016-05-17 21:52:40,157 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2/_temporary
2016-05-17 21:52:40,158 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  completed in: 00:00.401
2016-05-17 21:52:40,229 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-17 21:52:40,252 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:40,253 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:40,297 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:52:40,310 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-05-17 21:52:40,311 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:40,311 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:40,311 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:40,327 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 21:52:40,333 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-17 21:52:40,377 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 21:52:40,377 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,377 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-17 21:52:40,378 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  parallel execution of steps is enabled: false
2016-05-17 21:52:40,378 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  executing total steps: 1
2016-05-17 21:52:40,378 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  allocating management threads: 1
2016-05-17 21:52:40,378 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount3/count3
2016-05-17 21:52:40,379 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:40,391 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 21:52:40,395 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:40,395 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:40,395 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:40,404 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001 doesn't exist and is created
2016-05-17 21:52:40,420 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:40,420 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:40,420 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001
2016-05-17 21:52:40,423 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521960395_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:40,433 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:40,433 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:40,434 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:40,435 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:40,435 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001/localmode-log-dir
2016-05-17 21:52:40,435 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001/localmode-log-dir/history.txt.appattempt_1463521960395_0001_000000, maxErrors=10
2016-05-17 21:52:40,435 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:40,435 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521960395_0001_000000, appSubmitTime=1463521960423, launchTime=1463521960423
2016-05-17 21:52:40,441 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:40,442 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:40,442 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:40,442 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:40,448 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48877
2016-05-17 21:52:40,449 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:40,450 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521960395_0001_000000, startTime=1463521960450
2016-05-17 21:52:40,450 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:40,451 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:40,520 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:40,521 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:40,521 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463521960395_0001, dagName=[8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3
2016-05-17 21:52:40,537 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3
2016-05-17 21:52:40,538 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521960395_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001/localmode-log-dir/dag_1463521960395_0001_1.dot
2016-05-17 21:52:40,539 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001/localmode-log-dir/dag_1463521960395_0001_1-tez-dag.pb.txt
2016-05-17 21:52:40,546 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521960395_0001_1
2016-05-17 21:52:40,546 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,547 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3
2016-05-17 21:52:40 Running Dag: dag_1463521960395_0001_1
2016-05-17 21:52:40,559 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521960395_0001_1, submitTime=1463521960537
2016-05-17 21:52:40,559 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:40,559 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,560 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:40,560 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:40,560 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521960395_0001_1, initTime=1463521960559
2016-05-17 21:52:40,560 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960395_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:40,561 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_STARTED]: dagID=dag_1463521960395_0001_1, startTime=1463521960561
2016-05-17 21:52:40,561 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:40,561 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960395_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:40,561 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:40,561 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D8187D446EE8478FA794302BE36A878E : {A0EAA328737C44C684F185EDE31B2477={InputName=A0EAA328737C44C684F185EDE31B2477}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:40,562 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A0EAA328737C44C684F185EDE31B2477, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:40,562 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,562 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,562 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,562 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]. Starting root input initializers: 1
2016-05-17 21:52:40,563 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:40,563 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,563 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A0EAA328737C44C684F185EDE31B2477 on vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,564 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463521960395_0001, dagName=[8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3
2016-05-17 21:52:40,565 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:40,565 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:40,566 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,566 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A0EAA328737C44C684F185EDE31B2477 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:40,566 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,567 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,567 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FD21EC6DAF884AFABDA16FD56BF6A583, vertexId=vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:40,567 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:40,568 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FD21EC6DAF884AFABDA16FD56BF6A583, vertexId=vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,568 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:40,570 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 21:52:40,570 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FD21EC6DAF884AFABDA16FD56BF6A583, vertexName=A58EAD1A40BE475786E3B211FCAC832D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:40,571 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FD21EC6DAF884AFABDA16FD56BF6A583, vertexId=vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,570 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A0EAA328737C44C684F185EDE31B2477 on vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,572 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, vertexId=vertex_1463521960395_0001_1_01, initRequestedTime=1463521960563, initedTime=1463521960572, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:40,572 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:40,572 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:40,573 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:40,573 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A0EAA328737C44C684F185EDE31B2477=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:40,574 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] parallelism set to 1
2016-05-17 21:52:40,574 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:40,575 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521960395_0001_1_00, eventCount=1
2016-05-17 21:52:40,575 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D8187D446EE8478FA794302BE36A878E, vertexId=vertex_1463521960395_0001_1_00, initRequestedTime=1463521960561, initedTime=1463521960575, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:40,576 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] with distanceFromRoot: 0
2016-05-17 21:52:40,576 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:40,576 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:40,576 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,577 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,577 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:40,578 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521960395_0001_1_00, startRequestedTime=1463521960576, startedTime=1463521960576
2016-05-17 21:52:40,578 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:40,578 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:40,579 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_STARTED]: vertexName=D8187D446EE8478FA794302BE36A878E, taskId=task_1463521960395_0001_1_00_000000, scheduledTime=1463521960578, launchTime=1463521960578
2016-05-17 21:52:40,579 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:40,579 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521960395_0001_1_00 for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] numStartedSources: 1 numSources: 1
2016-05-17 21:52:40,579 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] with distanceFromRoot: 1
2016-05-17 21:52:40,580 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,580 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D8187D446EE8478FA794302BE36A878E in vertex: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,581 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A58EAD1A40BE475786E3B211FCAC832D with 1 source tasks and 1 pending tasks
2016-05-17 21:52:40,581 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:40,581 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521960395_0001_1_01, startRequestedTime=1463521960580, startedTime=1463521960580
2016-05-17 21:52:40,582 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:40,582 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3, VertexName: D8187D446EE8478FA794302BE36A878E, VertexParallelism: 1, TaskAttemptID:attempt_1463521960395_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A0EAA328737C44C684F185EDE31B2477, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A58EAD1A40BE475786E3B211FCAC832D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:40,583 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:40,583 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:40,584 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:40,585 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:40,585 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:40,586 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521960395_0001_00_000001
2016-05-17 21:52:40,586 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:40,588 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521960395_0001_00_000001
2016-05-17 21:52:40,588 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:40,588 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463521960395_0001, with dag id: dag_1463521960395_0001_1
2016-05-17 21:52:40,590 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:40,592 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:40,592 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521960395_0001_00_000001, launchTime=1463521960589
2016-05-17 21:52:40,593 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960395_0001_00_000001
2016-05-17 21:52:40,593 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521960395_0001_1_00_000000_0] to container: [container_1463521960395_0001_00_000001]
2016-05-17 21:52:40,593 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:40,594 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521960395_0001_1_00_000000_0] started. Is using containerId: [container_1463521960395_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:40,594 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D8187D446EE8478FA794302BE36A878E, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0, startTime=1463521960594, containerId=container_1463521960395_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521960395_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:40,594 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:40,594 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:40,594 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:40,595 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521960395_0001_00_000001 given task: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,595 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960395_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40 Starting to run new task attempt: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,596 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,596 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,596 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:40,597 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:40,597 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:40,597 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:40,598 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:40,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3, VertexName: D8187D446EE8478FA794302BE36A878E, VertexParallelism: 1, TaskAttemptID:attempt_1463521960395_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A0EAA328737C44C684F185EDE31B2477, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A58EAD1A40BE475786E3B211FCAC832D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:40,598 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:40,599 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:40,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,599 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521960395_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:40,601 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A0EAA328737C44C684F185EDE31B2477, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:40,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:40,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,603 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,604 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:40,605 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,605 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:40,605 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A58EAD1A40BE475786E3B211FCAC832D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:40,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:40,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,608 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:40,608 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,609 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,612 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D8187D446EE8478FA794302BE36A878E, ordinal: 0
2016-05-17 21:52:40,612 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8187D446EE8478FA794302BE36A878E, mem on start (mb), free: 325, total: 556, max: 672
2016-05-17 21:52:40,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:40,612 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:40,613 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:40,613 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:40,613 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:40,614 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:40,614 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:40,614 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:40,614 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:40,615 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A0EAA328737C44C684F185EDE31B2477, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:40,615 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A58EAD1A40BE475786E3B211FCAC832D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:40,615 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:40,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A0EAA328737C44C684F185EDE31B2477 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:40,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,616 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:40,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,616 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:40,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:40,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:40,618 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,619 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:40,619 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1de4b844
2016-05-17 21:52:40,620 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1de4b844
2016-05-17 21:52:40,620 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:40,621 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:40,621 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,622 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,622 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 0FC613E742CE4266A24609E9E8BE9972
2016-05-17 21:52:40,622 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 0FC613E742CE4266A24609E9E8BE9972
2016-05-17 21:52:40,624 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@748860cc; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@258b2cd9; comparator=cascading.tuple.hadoop.util.TupleComparator@542d6dd0; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:40,624 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:40,624 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:40,624 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:40,643 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:40,643 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:40,643 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:40,643 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:40,644 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:40,645 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,646 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:40,646 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D8187D446EE8478FA794302BE36A878E, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:40,648 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,649 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8187D446EE8478FA794302BE36A878E, mem on close (mb), free: 417, total: 548, max: 672
2016-05-17 21:52:40,649 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:40,649 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:40,649 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 21:52:40,649 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:52:40,653 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,656 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:40,656 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521960395_0001_1_00, eventCount=1
2016-05-17 21:52:40,657 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521960646, Process_Duration=2, Process_End_Time=1463521960648, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:52:40,657 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:40,657 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,657 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:40,657 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D8187D446EE8478FA794302BE36A878E, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0, startTime=1463521960594, finishTime=1463521960657, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960646, Process_Duration=2, Process_End_Time=1463521960648, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:40,658 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:40,658 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:40,657 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960395_0001_00_000001
2016-05-17 21:52:40,658 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,659 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:40,659 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_FINISHED]: vertexName=D8187D446EE8478FA794302BE36A878E, taskId=task_1463521960395_0001_1_00_000000, startTime=1463521960594, finishTime=1463521960659, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1463521960395_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960646, Process_Duration=2, Process_End_Time=1463521960648, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:40,660 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:40,660 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] : 1
2016-05-17 21:52:40,660 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,661 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:40,661 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_FINISHED]: vertexName=D8187D446EE8478FA794302BE36A878E, vertexId=vertex_1463521960395_0001_1_00, initRequestedTime=1463521960561, initedTime=1463521960575, startRequestedTime=1463521960576, startedTime=1463521960576, finishTime=1463521960660, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960646, Process_Duration=2, Process_End_Time=1463521960648, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463521960594, firstTasksToStart=[ task_1463521960395_0001_1_00_000000 ], lastTaskFinishTime=1463521960657, lastTasksToFinish=[ task_1463521960395_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521960395_0001_1_00_000000 ], longestDurationTasks=[ task_1463521960395_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:40,661 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:40,661 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:40,662 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521960395_0001_00_000001, stoppedTime=1463521960662, exitStatus=0
2016-05-17 21:52:40,662 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:40,662 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:40,662 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] attempt: attempt_1463521960395_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:40,662 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,662 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,663 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:40,663 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521960395_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:40,663 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_STARTED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, taskId=task_1463521960395_0001_1_01_000000, scheduledTime=1463521960663, launchTime=1463521960663
2016-05-17 21:52:40,663 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:40,663 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:40,663 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:40,664 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,664 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3, VertexName: A58EAD1A40BE475786E3B211FCAC832D, VertexParallelism: 1, TaskAttemptID:attempt_1463521960395_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D8187D446EE8478FA794302BE36A878E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FD21EC6DAF884AFABDA16FD56BF6A583, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:40,664 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:40,664 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:40,665 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521960395_0001_00_000002
2016-05-17 21:52:40,665 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:40,666 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521960395_0001_00_000002
2016-05-17 21:52:40,666 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:40,667 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:40,667 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:40,667 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521960395_0001_00_000002, launchTime=1463521960667
2016-05-17 21:52:40,668 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960395_0001_00_000002
2016-05-17 21:52:40,668 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521960395_0001_1_01_000000_0] to container: [container_1463521960395_0001_00_000002]
2016-05-17 21:52:40,668 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:40,669 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521960395_0001_00_000002 given task: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,669 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960395_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,669 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521960395_0001_1_01_000000_0] started. Is using containerId: [container_1463521960395_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:40 Starting to run new task attempt: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,669 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,669 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,670 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:40,670 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:40,670 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:40,670 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:40,670 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:40,670 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3, VertexName: A58EAD1A40BE475786E3B211FCAC832D, VertexParallelism: 1, TaskAttemptID:attempt_1463521960395_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D8187D446EE8478FA794302BE36A878E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FD21EC6DAF884AFABDA16FD56BF6A583, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:40,671 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:40,671 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521960395_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:40,671 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960395_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:40,672 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521960395_0001_00_000001, Exiting
2016-05-17 21:52:40,672 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521960395_0001_00_000001
2016-05-17 21:52:40,672 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521960395_0001_00_000001
2016-05-17 21:52:40,670 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0, startTime=1463521960669, containerId=container_1463521960395_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521960395_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:40,670 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:40,674 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:40,672 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,674 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:40,675 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,672 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521960395_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:40,676 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:40,675 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521960395_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:40,676 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,676 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FD21EC6DAF884AFABDA16FD56BF6A583, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:40,677 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:40,677 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FD21EC6DAF884AFABDA16FD56BF6A583
2016-05-17 21:52:40,677 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:40,678 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FD21EC6DAF884AFABDA16FD56BF6A583
2016-05-17 21:52:40,677 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D8187D446EE8478FA794302BE36A878E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:40,679 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:40,679 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,679 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521960395_0001_00_000001 completed successfully
2016-05-17 21:52:40,679 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:40,681 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:40,682 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:40,682 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,684 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:40,685 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,692 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:40,692 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FD21EC6DAF884AFABDA16FD56BF6A583
2016-05-17 21:52:40,693 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A58EAD1A40BE475786E3B211FCAC832D, ordinal: 1
2016-05-17 21:52:40,693 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A58EAD1A40BE475786E3B211FCAC832D, mem on start (mb), free: 410, total: 548, max: 672
2016-05-17 21:52:40,694 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,694 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:40,694 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:40,695 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:40,695 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:40,696 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:40,696 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:40,696 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:40,697 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:40,697 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:40,698 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FD21EC6DAF884AFABDA16FD56BF6A583, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:40,698 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D8187D446EE8478FA794302BE36A878E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:40,698 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:40,699 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,700 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:40,700 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:40,700 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D8187D446EE8478FA794302BE36A878E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:40,700 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:40,700 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:40,701 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:40,701 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,704 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D8187D446EE8478FA794302BE36A878E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:40,704 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:40,704 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:40,705 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:40,705 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,705 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521960395_0001_1_00_000000_0_10074, runDuration: 47821, ]
2016-05-17 21:52:40,707 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521960395_0001_1_00_000000_0_10074] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463521960707, TimeTaken=1, Rate=0.44 MB/s
2016-05-17 21:52:40,707 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,707 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:40,708 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D8187D446EE8478FA794302BE36A878E] #12 in 2ms
2016-05-17 21:52:40,708 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D8187D446EE8478FA794302BE36A878E] #12, status:false, isInterrupted:false
2016-05-17 21:52:40,709 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:40,709 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 21:52:40,710 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:40,710 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:40,710 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 21:52:40,710 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,710 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:40,713 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 21:52:40,713 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 0FC613E742CE4266A24609E9E8BE9972
2016-05-17 21:52:40,714 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"], id: FD21EC6DAF884AFABDA16FD56BF6A583
2016-05-17 21:52:40,714 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-17 21:52:40,714 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 0FC613E742CE4266A24609E9E8BE9972, for 1 inputs
2016-05-17 21:52:40,715 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A58EAD1A40BE475786E3B211FCAC832D, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:40,717 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@29ef44ef; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@1b96bc6b
2016-05-17 21:52:40,720 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:40,721 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:40,723 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,723 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521960395_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:40,724 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,724 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A58EAD1A40BE475786E3B211FCAC832D, mem on close (mb), free: 408, total: 548, max: 672
2016-05-17 21:52:40,725 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,725 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:40,726 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:40,726 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:40,726 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:40,727 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,728 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:40,729 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:40,729 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0, startTime=1463521960669, finishTime=1463521960729, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960715, Process_Duration=9, Process_End_Time=1463521960724, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:40,730 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:40,730 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,730 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_FINISHED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, taskId=task_1463521960395_0001_1_01_000000, startTime=1463521960669, finishTime=1463521960730, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463521960395_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960715, Process_Duration=9, Process_End_Time=1463521960724, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:40,731 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:40,731 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] : 1
2016-05-17 21:52:40,731 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,731 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:40,732 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:40,732 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:40,729 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521960715, Process_Duration=9, Process_End_Time=1463521960724, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 21:52:40,732 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_FINISHED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, vertexId=vertex_1463521960395_0001_1_01, initRequestedTime=1463521960563, initedTime=1463521960572, startRequestedTime=1463521960580, startedTime=1463521960580, finishTime=1463521960731, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960715, Process_Duration=9, Process_End_Time=1463521960724, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463521960669, firstTasksToStart=[ task_1463521960395_0001_1_01_000000 ], lastTaskFinishTime=1463521960729, lastTasksToFinish=[ task_1463521960395_0001_1_01_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521960395_0001_1_01_000000 ], longestDurationTasks=[ task_1463521960395_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:40,733 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:40,733 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:40,733 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:40,734 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,734 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521960395_0001_00_000002, stoppedTime=1463521960734, exitStatus=0
2016-05-17 21:52:40,734 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960395_0001_00_000002
2016-05-17 21:52:40,734 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:40,734 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521960395_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:40,734 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:40,734 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960395_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:40,734 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:40,735 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521960395_0001_00_000002, Exiting
2016-05-17 21:52:40,735 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521960395_0001_00_000002
2016-05-17 21:52:40,735 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521960395_0001_00_000002
2016-05-17 21:52:40,735 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521960395_0001_1
2016-05-17 21:52:40,735 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521960395_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:40,735 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521960395_0001_00_000002 completed successfully
2016-05-17 21:52:40,736 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521960395_0001_1
2016-05-17 21:52:40,736 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,736 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: FD21EC6DAF884AFABDA16FD56BF6A583 for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,746 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521960395_0001_1, queueSize=0
2016-05-17 21:52:40,746 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:40,746 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521960395_0001_1, startTime=1463521960561, finishTime=1463521960744, timeTaken=183, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1149239296, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043921361, Process_Duration=11, Process_End_Time=2927043921372, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 21:52:40,746 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521960395_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:40,746 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960395_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:40,746 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521960395_0001_00_000002
2016-05-17 21:52:40,746 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:40 Completed Dag: dag_1463521960395_0001_1
2016-05-17 21:52:40,747 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521960395_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:40,747 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521960395_0001_1_post
2016-05-17 21:52:40,748 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,748 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:40,753 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463521960395_0001
2016-05-17 21:52:40,754 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:40,755 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:40,755 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:40,755 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:40,755 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:40,756 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:40,756 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:40,756 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:40,756 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:40,757 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:40,757 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:40,757 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:40,757 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:40,762 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001
2016-05-17 21:52:40,763 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3/_temporary
2016-05-17 21:52:40,763 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  completed in: 00:00.385
2016-05-17 21:52:40,816 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:40,816 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:40,831 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-17 21:52:40,854 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:40,855 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:40,887 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:52:40,895 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 21:52:40,896 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:40,896 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:40,896 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:40,910 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 21:52:40,916 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-17 21:52:40,959 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 21:52:40,959 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,960 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-17 21:52:40,960 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  parallel execution of steps is enabled: false
2016-05-17 21:52:40,960 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  executing total steps: 1
2016-05-17 21:52:40,960 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  allocating management threads: 1
2016-05-17 21:52:40,961 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount4/count4
2016-05-17 21:52:40,961 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:40,973 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 21:52:40,977 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:40,978 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:40,978 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:40,987 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001 doesn't exist and is created
2016-05-17 21:52:41,003 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:41,003 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:41,004 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001
2016-05-17 21:52:41,006 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521960977_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:41,016 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:41,017 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:41,018 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:41,018 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:41,019 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001/localmode-log-dir
2016-05-17 21:52:41,019 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001/localmode-log-dir/history.txt.appattempt_1463521960977_0001_000000, maxErrors=10
2016-05-17 21:52:41,019 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:41,019 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521960977_0001_000000, appSubmitTime=1463521961006, launchTime=1463521961006
2016-05-17 21:52:41,025 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:41,028 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:41,029 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58479
2016-05-17 21:52:41,029 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:41,029 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:41,032 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:41,034 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521960977_0001_000000, startTime=1463521961034
2016-05-17 21:52:41,034 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:41,034 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:41,104 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:41,104 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:41,104 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463521960977_0001, dagName=[52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4
2016-05-17 21:52:41,120 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4
2016-05-17 21:52:41,121 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521960977_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001/localmode-log-dir/dag_1463521960977_0001_1.dot
2016-05-17 21:52:41,122 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001/localmode-log-dir/dag_1463521960977_0001_1-tez-dag.pb.txt
2016-05-17 21:52:41,130 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521960977_0001_1
2016-05-17 21:52:41,130 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,130 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4
2016-05-17 21:52:41 Running Dag: dag_1463521960977_0001_1
2016-05-17 21:52:41,141 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521960977_0001_1, submitTime=1463521961120
2016-05-17 21:52:41,141 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:41,142 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,142 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:41,142 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:41,142 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521960977_0001_1, initTime=1463521961141
2016-05-17 21:52:41,143 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960977_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:41,143 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_STARTED]: dagID=dag_1463521960977_0001_1, startTime=1463521961143
2016-05-17 21:52:41,143 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:41,144 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:41,144 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960977_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:41,144 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EC8306BF07FC4B23939584C6A74A43C0 : {BD188B76DC47455FB30B8AEB61242335={InputName=BD188B76DC47455FB30B8AEB61242335}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:41,144 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BD188B76DC47455FB30B8AEB61242335, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:41,144 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463521960977_0001, dagName=[52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4
2016-05-17 21:52:41,144 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,145 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,145 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,145 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]. Starting root input initializers: 1
2016-05-17 21:52:41,146 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:41,146 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BD188B76DC47455FB30B8AEB61242335 on vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,146 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,148 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:41,149 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BD188B76DC47455FB30B8AEB61242335 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:41,149 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:41,149 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:41,150 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,150 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,150 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,151 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, vertexId=vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:41,151 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, vertexId=vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,150 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:41,153 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:41,153 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, vertexId=vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,154 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 21:52:41,155 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BD188B76DC47455FB30B8AEB61242335 on vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,155 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:41,156 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, vertexId=vertex_1463521960977_0001_1_01, initRequestedTime=1463521961146, initedTime=1463521961155, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:41,156 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:41,156 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:41,156 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BD188B76DC47455FB30B8AEB61242335=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:41,156 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] parallelism set to 1
2016-05-17 21:52:41,156 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:41,157 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521960977_0001_1_00, eventCount=1
2016-05-17 21:52:41,157 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:41,157 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, vertexId=vertex_1463521960977_0001_1_00, initRequestedTime=1463521961144, initedTime=1463521961157, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:41,157 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] with distanceFromRoot: 0
2016-05-17 21:52:41,157 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:41,158 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,158 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,158 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:41,158 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521960977_0001_1_00, startRequestedTime=1463521961158, startedTime=1463521961158
2016-05-17 21:52:41,158 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:41,159 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:41,159 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_STARTED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, taskId=task_1463521960977_0001_1_00_000000, scheduledTime=1463521961159, launchTime=1463521961159
2016-05-17 21:52:41,159 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:41,159 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521960977_0001_1_00 for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] numStartedSources: 1 numSources: 1
2016-05-17 21:52:41,159 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] with distanceFromRoot: 1
2016-05-17 21:52:41,160 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,160 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EC8306BF07FC4B23939584C6A74A43C0 in vertex: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,160 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B3A5359E5D9546B7B4AD5BD3BBA1F66F with 1 source tasks and 1 pending tasks
2016-05-17 21:52:41,160 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:41,161 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521960977_0001_1_01, startRequestedTime=1463521961160, startedTime=1463521961160
2016-05-17 21:52:41,161 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:41,161 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4, VertexName: EC8306BF07FC4B23939584C6A74A43C0, VertexParallelism: 1, TaskAttemptID:attempt_1463521960977_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD188B76DC47455FB30B8AEB61242335, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:41,162 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:41,162 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:41,162 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:41,163 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:41,163 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:41,164 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521960977_0001_00_000001
2016-05-17 21:52:41,164 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:41,166 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521960977_0001_00_000001
2016-05-17 21:52:41,166 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:41,167 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:41,167 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:41,167 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521960977_0001_00_000001, launchTime=1463521961167
2016-05-17 21:52:41,167 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463521960977_0001, with dag id: dag_1463521960977_0001_1
2016-05-17 21:52:41,170 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960977_0001_00_000001
2016-05-17 21:52:41,171 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521960977_0001_1_00_000000_0] to container: [container_1463521960977_0001_00_000001]
2016-05-17 21:52:41,171 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:41,171 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521960977_0001_00_000001 given task: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,171 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960977_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,174 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521960977_0001_1_00_000000_0] started. Is using containerId: [container_1463521960977_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:41,175 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0, startTime=1463521961172, containerId=container_1463521960977_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521960977_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:41,175 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:41,175 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:41,175 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:41 Starting to run new task attempt: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,176 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,176 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,176 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:41,176 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:41,176 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:41,176 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:41,178 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:41,178 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4, VertexName: EC8306BF07FC4B23939584C6A74A43C0, VertexParallelism: 1, TaskAttemptID:attempt_1463521960977_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD188B76DC47455FB30B8AEB61242335, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:41,179 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:41,180 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,180 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:41,180 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,181 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521960977_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:41,181 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BD188B76DC47455FB30B8AEB61242335, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:41,181 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:41,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:41,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:41,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,182 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,184 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:41,186 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:41,186 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,189 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:41,189 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,189 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,190 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EC8306BF07FC4B23939584C6A74A43C0, ordinal: 0
2016-05-17 21:52:41,190 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EC8306BF07FC4B23939584C6A74A43C0, mem on start (mb), free: 318, total: 548, max: 672
2016-05-17 21:52:41,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:41,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:41,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:41,191 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:41,192 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:41,192 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:41,192 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:41,192 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:41,192 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:41,193 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BD188B76DC47455FB30B8AEB61242335, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:41,193 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B3A5359E5D9546B7B4AD5BD3BBA1F66F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:41,193 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:41,193 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BD188B76DC47455FB30B8AEB61242335 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:41,193 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,193 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:41,194 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,193 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:41,194 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:41,194 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:41,194 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,194 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:41,195 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@73e8f917
2016-05-17 21:52:41,196 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@73e8f917
2016-05-17 21:52:41,196 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:41,196 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:41,198 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:41,198 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,198 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 2EE1A62750F64CE0888F2B9991F60303
2016-05-17 21:52:41,199 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 2EE1A62750F64CE0888F2B9991F60303
2016-05-17 21:52:41,201 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@21a7333c; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6814b966; comparator=cascading.tuple.hadoop.util.TupleComparator@28980cec; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:41,202 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:41,202 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:41,202 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:41,223 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:41,223 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:41,224 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:41,224 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:41,224 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:41,224 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:41,224 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:41,225 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EC8306BF07FC4B23939584C6A74A43C0, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:41,228 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,228 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EC8306BF07FC4B23939584C6A74A43C0, mem on close (mb), free: 423, total: 555, max: 672
2016-05-17 21:52:41,229 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:41,229 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:41,229 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 21:52:41,229 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:52:41,233 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:41 Completed running task attempt: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,236 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:41,237 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521960977_0001_1_00, eventCount=1
2016-05-17 21:52:41,237 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:41,237 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0, startTime=1463521961172, finishTime=1463521961237, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961225, Process_Duration=3, Process_End_Time=1463521961228, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:41,237 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:41,238 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,238 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:41,238 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:41,238 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_FINISHED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, taskId=task_1463521960977_0001_1_00_000000, startTime=1463521961172, finishTime=1463521961238, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1463521960977_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961225, Process_Duration=3, Process_End_Time=1463521961228, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:41,238 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:41,239 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] : 1
2016-05-17 21:52:41,239 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,240 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:41,240 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_FINISHED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, vertexId=vertex_1463521960977_0001_1_00, initRequestedTime=1463521961144, initedTime=1463521961157, startRequestedTime=1463521961158, startedTime=1463521961158, finishTime=1463521961239, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961225, Process_Duration=3, Process_End_Time=1463521961228, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463521961172, firstTasksToStart=[ task_1463521960977_0001_1_00_000000 ], lastTaskFinishTime=1463521961237, lastTasksToFinish=[ task_1463521960977_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521960977_0001_1_00_000000 ], longestDurationTasks=[ task_1463521960977_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:41,240 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:41,240 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:41,241 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] attempt: attempt_1463521960977_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:41,241 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,241 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,241 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521960977_0001_00_000001, stoppedTime=1463521961241, exitStatus=0
2016-05-17 21:52:41,241 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:41,241 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:41,242 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:41,242 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_STARTED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, taskId=task_1463521960977_0001_1_01_000000, scheduledTime=1463521961242, launchTime=1463521961242
2016-05-17 21:52:41,242 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:41,242 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521961225, Process_Duration=3, Process_End_Time=1463521961228, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:52:41,242 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:41,243 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521960977_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:41,243 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:41,243 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,243 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,243 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:41,243 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4, VertexName: B3A5359E5D9546B7B4AD5BD3BBA1F66F, VertexParallelism: 1, TaskAttemptID:attempt_1463521960977_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EC8306BF07FC4B23939584C6A74A43C0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:41,244 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:41,244 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:41,244 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960977_0001_00_000001
2016-05-17 21:52:41,244 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521960977_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:41,245 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960977_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:41,245 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521960977_0001_00_000002
2016-05-17 21:52:41,245 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:41,246 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521960977_0001_00_000001, Exiting
2016-05-17 21:52:41,246 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521960977_0001_00_000001
2016-05-17 21:52:41,246 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521960977_0001_00_000001
2016-05-17 21:52:41,247 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521960977_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:41,247 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521960977_0001_00_000002
2016-05-17 21:52:41,247 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521960977_0001_00_000001 completed successfully
2016-05-17 21:52:41,248 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:41,248 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:41,250 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:41,250 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:41,251 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521960977_0001_00_000002, launchTime=1463521961250
2016-05-17 21:52:41,251 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960977_0001_00_000002
2016-05-17 21:52:41,252 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521960977_0001_1_01_000000_0] to container: [container_1463521960977_0001_00_000002]
2016-05-17 21:52:41,252 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:41,252 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521960977_0001_00_000002 given task: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,253 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960977_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,253 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521960977_0001_1_01_000000_0] started. Is using containerId: [container_1463521960977_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:41,253 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:41,253 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0, startTime=1463521961252, containerId=container_1463521960977_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521960977_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:41,253 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:41,254 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:41 Starting to run new task attempt: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,254 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,255 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,255 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:41,255 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:41,255 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:41,256 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:41,256 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:41,256 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4, VertexName: B3A5359E5D9546B7B4AD5BD3BBA1F66F, VertexParallelism: 1, TaskAttemptID:attempt_1463521960977_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EC8306BF07FC4B23939584C6A74A43C0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:41,256 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:41,257 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,257 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:41,257 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EC8306BF07FC4B23939584C6A74A43C0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:41,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:41,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,260 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:41,260 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:41,260 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,260 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,261 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,261 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521960977_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:41,263 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:41,264 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:41,264 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:41,265 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F
2016-05-17 21:52:41,266 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:41,266 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F
2016-05-17 21:52:41,270 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B3A5359E5D9546B7B4AD5BD3BBA1F66F, ordinal: 1
2016-05-17 21:52:41,270 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3A5359E5D9546B7B4AD5BD3BBA1F66F, mem on start (mb), free: 417, total: 555, max: 672
2016-05-17 21:52:41,270 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,271 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:41,271 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:41,276 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:41,277 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F
2016-05-17 21:52:41,277 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:41,277 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:41,277 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:41,278 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:41,278 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:41,278 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:41,278 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:41,279 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EC8306BF07FC4B23939584C6A74A43C0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:41,279 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 25FDD7E0D74A4B7D90BB4AB1B8B6D84F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:41,279 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:41,279 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,280 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EC8306BF07FC4B23939584C6A74A43C0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:41,280 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:41,280 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:41,281 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:41,281 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:41,282 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EC8306BF07FC4B23939584C6A74A43C0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:41,282 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:41,282 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:41,282 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:41,283 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,283 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:41,284 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,287 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 21:52:41,287 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521960977_0001_1_00_000000_0_10080, runDuration: 47675, ]
2016-05-17 21:52:41,287 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 2EE1A62750F64CE0888F2B9991F60303
2016-05-17 21:52:41,288 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"], id: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F
2016-05-17 21:52:41,288 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-17 21:52:41,289 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 2EE1A62750F64CE0888F2B9991F60303, for 1 inputs
2016-05-17 21:52:41,289 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521960977_0001_1_00_000000_0_10080] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463521961289, TimeTaken=1, Rate=0.44 MB/s
2016-05-17 21:52:41,290 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,290 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:41,290 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [EC8306BF07FC4B23939584C6A74A43C0] #13 in 2ms
2016-05-17 21:52:41,290 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EC8306BF07FC4B23939584C6A74A43C0] #13, status:false, isInterrupted:false
2016-05-17 21:52:41,291 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:41,292 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 21:52:41,292 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:41,292 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:41,292 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 21:52:41,293 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,293 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B3A5359E5D9546B7B4AD5BD3BBA1F66F, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:52:41,293 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:41,294 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1c20e05c; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@3269c671
2016-05-17 21:52:41,297 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:41,297 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:41,298 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,298 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521960977_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:41,298 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,299 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3A5359E5D9546B7B4AD5BD3BBA1F66F, mem on close (mb), free: 414, total: 555, max: 672
2016-05-17 21:52:41,299 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,299 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:41,300 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:41,300 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:41,300 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:41,300 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:41 Completed running task attempt: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,301 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:41,302 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521961293, Process_Duration=5, Process_End_Time=1463521961298, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 21:52:41,302 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0, startTime=1463521961252, finishTime=1463521961302, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961293, Process_Duration=5, Process_End_Time=1463521961298, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:41,303 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:41,303 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:41,303 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,302 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:41,303 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:41,304 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:41,304 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,304 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960977_0001_00_000002
2016-05-17 21:52:41,303 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_FINISHED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, taskId=task_1463521960977_0001_1_01_000000, startTime=1463521961252, finishTime=1463521961303, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463521960977_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961293, Process_Duration=5, Process_End_Time=1463521961298, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:41,305 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:41,305 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] : 1
2016-05-17 21:52:41,305 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,306 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:41,307 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_FINISHED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, vertexId=vertex_1463521960977_0001_1_01, initRequestedTime=1463521961146, initedTime=1463521961155, startRequestedTime=1463521961160, startedTime=1463521961160, finishTime=1463521961305, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961293, Process_Duration=5, Process_End_Time=1463521961298, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463521961252, firstTasksToStart=[ task_1463521960977_0001_1_01_000000 ], lastTaskFinishTime=1463521961302, lastTasksToFinish=[ task_1463521960977_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521960977_0001_1_01_000000 ], longestDurationTasks=[ task_1463521960977_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:41,307 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:41,307 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:41,307 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521960977_0001_00_000002, stoppedTime=1463521961307, exitStatus=0
2016-05-17 21:52:41,307 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:41,307 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:41,308 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:41,308 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521960977_0001_1
2016-05-17 21:52:41,308 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521960977_0001_1
2016-05-17 21:52:41,308 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,315 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521960977_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:41,315 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960977_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:41,315 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521960977_0001_00_000002, Exiting
2016-05-17 21:52:41,315 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521960977_0001_00_000002
2016-05-17 21:52:41,315 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521960977_0001_00_000002
2016-05-17 21:52:41,316 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521960977_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:41,316 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521960977_0001_00_000002 completed successfully
2016-05-17 21:52:41,316 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,319 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521960977_0001_1, queueSize=0
2016-05-17 21:52:41,319 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:41,319 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521960977_0001_1, startTime=1463521961143, finishTime=1463521961316, timeTaken=173, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1164967936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043922518, Process_Duration=8, Process_End_Time=2927043922526, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 21:52:41,320 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521960977_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:41,320 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960977_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:41,320 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521960977_0001_00_000002
2016-05-17 21:52:41,320 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:41 Completed Dag: dag_1463521960977_0001_1
2016-05-17 21:52:41,320 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521960977_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:41,320 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521960977_0001_1_post
2016-05-17 21:52:41,321 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,321 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:41,326 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463521960977_0001
2016-05-17 21:52:41,327 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:41,328 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:41,328 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:41,328 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:41,328 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:41,329 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:41,329 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:41,329 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:41,329 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:41,329 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:41,330 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:41,330 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:41,330 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:41,334 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001
2016-05-17 21:52:41,334 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4/_temporary
2016-05-17 21:52:41,335 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  completed in: 00:00.375
2016-05-17 21:52:41,404 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-17 21:52:41,430 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:41,430 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:41,463 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:52:41,467 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:52:41,468 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:41,468 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:41,468 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:41,482 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:41,483 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-17 21:52:41,492 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:41,492 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:41,492 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-17 21:52:41,492 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:41,492 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:41,492 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:41,493 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testFilter/filter
2016-05-17 21:52:41,493 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:41,504 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:41,509 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:41,509 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:41,510 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:41,519 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001 doesn't exist and is created
2016-05-17 21:52:41,535 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:41,535 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:41,536 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001
2016-05-17 21:52:41,538 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521961509_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:41,548 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:41,548 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:41,549 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:41,550 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:41,550 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001/localmode-log-dir
2016-05-17 21:52:41,550 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001/localmode-log-dir/history.txt.appattempt_1463521961509_0001_000000, maxErrors=10
2016-05-17 21:52:41,550 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:41,550 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521961509_0001_000000, appSubmitTime=1463521961538, launchTime=1463521961538
2016-05-17 21:52:41,555 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:41,556 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:41,557 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54164
2016-05-17 21:52:41,558 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:41,558 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:41,559 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:41,561 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521961509_0001_000000, startTime=1463521961561
2016-05-17 21:52:41,561 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:41,561 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:41,635 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:41,635 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:41,636 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463521961509_0001, dagName=[7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter
2016-05-17 21:52:41,654 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter
2016-05-17 21:52:41,655 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521961509_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001/localmode-log-dir/dag_1463521961509_0001_1.dot
2016-05-17 21:52:41,655 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001/localmode-log-dir/dag_1463521961509_0001_1-tez-dag.pb.txt
2016-05-17 21:52:41,664 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521961509_0001_1
2016-05-17 21:52:41,665 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,665 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter
2016-05-17 21:52:41 Running Dag: dag_1463521961509_0001_1
2016-05-17 21:52:41,680 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:41,680 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:41,681 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521961509_0001_1, submitTime=1463521961654
2016-05-17 21:52:41,681 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:41,682 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CDED5065C842484FB462A32D3167D312
2016-05-17 21:52:41,682 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:41,683 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521961509_0001_1, initTime=1463521961681
2016-05-17 21:52:41,683 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:41,683 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961509_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:41,683 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_STARTED]: dagID=dag_1463521961509_0001_1, startTime=1463521961683
2016-05-17 21:52:41,683 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:41,683 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:41,684 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961509_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:41,684 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CDED5065C842484FB462A32D3167D312 : {33C1836C4FBB410A988935BB2DAFEA35={InputName=33C1836C4FBB410A988935BB2DAFEA35}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:41,684 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 33C1836C4FBB410A988935BB2DAFEA35, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:41,684 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,684 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463521961509_0001, dagName=[7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter
2016-05-17 21:52:41,685 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,685 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,685 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]. Starting root input initializers: 1
2016-05-17 21:52:41,685 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:41,686 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 33C1836C4FBB410A988935BB2DAFEA35 on vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,687 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:41,687 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 33C1836C4FBB410A988935BB2DAFEA35 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:41,687 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:41,688 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:41,689 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:41,689 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 33C1836C4FBB410A988935BB2DAFEA35 on vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,690 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:41,690 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {33C1836C4FBB410A988935BB2DAFEA35=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:41,690 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] parallelism set to 1
2016-05-17 21:52:41,690 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:41,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521961509_0001_1_00, eventCount=1
2016-05-17 21:52:41,691 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,691 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=817EF10BD9EE44FE91419FE44ACE46C5, vertexId=vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:41,691 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=817EF10BD9EE44FE91419FE44ACE46C5, vertexId=vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,693 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=817EF10BD9EE44FE91419FE44ACE46C5, vertexName=CDED5065C842484FB462A32D3167D312, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:41,693 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=817EF10BD9EE44FE91419FE44ACE46C5, vertexId=vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,694 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CDED5065C842484FB462A32D3167D312, vertexId=vertex_1463521961509_0001_1_00, initRequestedTime=1463521961684, initedTime=1463521961694, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:41,694 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] with distanceFromRoot: 0
2016-05-17 21:52:41,694 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:41,695 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:41,695 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CDED5065C842484FB462A32D3167D312
2016-05-17 21:52:41,695 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,696 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:41,696 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521961509_0001_1_00, startRequestedTime=1463521961695, startedTime=1463521961695
2016-05-17 21:52:41,696 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:41,697 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:41,697 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:TASK_STARTED]: vertexName=CDED5065C842484FB462A32D3167D312, taskId=task_1463521961509_0001_1_00_000000, scheduledTime=1463521961696, launchTime=1463521961696
2016-05-17 21:52:41,697 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961509_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:41,697 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,698 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter, VertexName: CDED5065C842484FB462A32D3167D312, VertexParallelism: 1, TaskAttemptID:attempt_1463521961509_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=33C1836C4FBB410A988935BB2DAFEA35, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=817EF10BD9EE44FE91419FE44ACE46C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:41,698 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961509_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:41,698 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:41,699 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:41,699 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:41,699 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:41,700 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521961509_0001_00_000001
2016-05-17 21:52:41,700 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521961509_0001, with dag id: dag_1463521961509_0001_1
2016-05-17 21:52:41,700 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:41,702 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521961509_0001_00_000001
2016-05-17 21:52:41,702 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:41,703 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:41,703 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:41,703 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521961509_0001_00_000001, launchTime=1463521961703
2016-05-17 21:52:41,704 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961509_0001_00_000001
2016-05-17 21:52:41,706 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521961509_0001_1_00_000000_0] to container: [container_1463521961509_0001_00_000001]
2016-05-17 21:52:41,706 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:41,706 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521961509_0001_1_00_000000_0] started. Is using containerId: [container_1463521961509_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:41,706 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CDED5065C842484FB462A32D3167D312, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0, startTime=1463521961706, containerId=container_1463521961509_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521961509_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:41,707 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:41,707 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521961509_0001_00_000001 given task: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,707 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961509_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:41,707 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961509_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:41,707 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961509_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41 Starting to run new task attempt: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,708 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,708 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,708 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:41,708 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:41,708 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:41,708 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:41,708 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:41,709 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter, VertexName: CDED5065C842484FB462A32D3167D312, VertexParallelism: 1, TaskAttemptID:attempt_1463521961509_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=33C1836C4FBB410A988935BB2DAFEA35, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=817EF10BD9EE44FE91419FE44ACE46C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:41,709 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:41,709 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,709 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:41,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,710 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521961509_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:41,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=33C1836C4FBB410A988935BB2DAFEA35, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:41,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=817EF10BD9EE44FE91419FE44ACE46C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:41,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:41,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:41,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,712 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 817EF10BD9EE44FE91419FE44ACE46C5
2016-05-17 21:52:41,712 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:41,712 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 817EF10BD9EE44FE91419FE44ACE46C5
2016-05-17 21:52:41,713 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:41,714 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:41,714 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,714 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,721 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:41,721 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 817EF10BD9EE44FE91419FE44ACE46C5
2016-05-17 21:52:41,722 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CDED5065C842484FB462A32D3167D312, ordinal: 0
2016-05-17 21:52:41,722 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDED5065C842484FB462A32D3167D312, mem on start (mb), free: 341, total: 555, max: 672
2016-05-17 21:52:41,722 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:41,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:41,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:41,723 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:41,723 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:41,723 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:41,723 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:41,723 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:41,724 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:41,724 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:41,724 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 33C1836C4FBB410A988935BB2DAFEA35, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:41,724 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 817EF10BD9EE44FE91419FE44ACE46C5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:41,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:41,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 33C1836C4FBB410A988935BB2DAFEA35 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:41,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,725 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:41,725 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,724 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:41,725 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:41,725 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:41,725 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,725 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:41,726 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@250eaa67
2016-05-17 21:52:41,727 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@250eaa67
2016-05-17 21:52:41,727 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:41,727 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:41,730 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:41,730 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,730 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"], id: 817EF10BD9EE44FE91419FE44ACE46C5
2016-05-17 21:52:41,731 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-17 21:52:41,731 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:41,731 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:41,731 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CDED5065C842484FB462A32D3167D312, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:41,733 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:41,733 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:41,734 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,734 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521961509_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:41,735 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,735 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDED5065C842484FB462A32D3167D312, mem on close (mb), free: 339, total: 555, max: 672
2016-05-17 21:52:41 Completed running task attempt: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,735 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:41,737 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CDED5065C842484FB462A32D3167D312, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0, startTime=1463521961706, finishTime=1463521961736, timeTaken=30, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 21:52:41,737 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:41,737 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961509_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:41,737 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:41,737 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,738 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:41,738 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:TASK_FINISHED]: vertexName=CDED5065C842484FB462A32D3167D312, taskId=task_1463521961509_0001_1_00_000000, startTime=1463521961706, finishTime=1463521961737, timeTaken=31, status=SUCCEEDED, successfulAttemptID=attempt_1463521961509_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 21:52:41,738 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961509_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:41,738 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] : 1
2016-05-17 21:52:41,738 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,739 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=3]]
2016-05-17 21:52:41,739 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:41,739 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,740 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961509_0001_00_000001
2016-05-17 21:52:41,741 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:VERTEX_FINISHED]: vertexName=CDED5065C842484FB462A32D3167D312, vertexId=vertex_1463521961509_0001_1_00, initRequestedTime=1463521961684, initedTime=1463521961694, startRequestedTime=1463521961695, startedTime=1463521961695, finishTime=1463521961738, timeTaken=43, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3, vertexStats=firstTaskStartTime=1463521961706, firstTasksToStart=[ task_1463521961509_0001_1_00_000000 ], lastTaskFinishTime=1463521961736, lastTasksToFinish=[ task_1463521961509_0001_1_00_000000 ], minTaskDuration=30, maxTaskDuration=30, avgTaskDuration=30.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521961509_0001_1_00_000000 ], longestDurationTasks=[ task_1463521961509_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:41,741 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:41,741 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:41,741 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521961509_0001_00_000001, stoppedTime=1463521961741, exitStatus=0
2016-05-17 21:52:41,741 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:41,741 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:41,741 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:41,741 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521961509_0001_1
2016-05-17 21:52:41,742 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521961509_0001_1
2016-05-17 21:52:41,742 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 817EF10BD9EE44FE91419FE44ACE46C5 for vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,742 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:41,750 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521961509_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:41,750 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961509_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:41,750 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521961509_0001_00_000001, Exiting
2016-05-17 21:52:41,750 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521961509_0001_00_000001
2016-05-17 21:52:41,750 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521961509_0001_00_000001
2016-05-17 21:52:41,752 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521961509_0001_1, queueSize=0
2016-05-17 21:52:41,752 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:41,752 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521961509_0001_1, startTime=1463521961683, finishTime=1463521961751, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 21:52:41,752 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521961509_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:41,752 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961509_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:41 Completed Dag: dag_1463521961509_0001_1
2016-05-17 21:52:41,753 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521961509_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:41,753 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521961509_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:41,753 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521961509_0001_1_post
2016-05-17 21:52:41,753 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,753 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:41,753 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:41,756 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521961509_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:41,756 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521961509_0001_00_000001 completed successfully
2016-05-17 21:52:41,757 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:41,759 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463521961509_0001
2016-05-17 21:52:41,761 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:41,761 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:41,761 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:41,761 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:41,762 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:41,762 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:41,762 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:41,762 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:41,763 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:41,763 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:41,763 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:41,763 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:41,763 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:41,768 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001
2016-05-17 21:52:41,768 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter/_temporary
2016-05-17 21:52:41,769 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.276
2016-05-17 21:52:41,810 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-17 21:52:41,836 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:41,837 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:41,875 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:52:41,889 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-17 21:52:41,889 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:41,890 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:41,890 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:41,905 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:41,912 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple
2016-05-17 21:52:41,973 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:41,973 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:41,973 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-17 21:52:41,974 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 21:52:41,974 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 21:52:41,974 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 21:52:41,975 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testSimple/simple
2016-05-17 21:52:41,975 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:41,987 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:41,991 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:41,992 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:41,992 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:42,001 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001 doesn't exist and is created
2016-05-17 21:52:42,016 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:42,016 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:42,017 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001
2016-05-17 21:52:42,020 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521961992_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:42,031 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:42,032 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:42,034 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:42,034 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:42,035 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001/localmode-log-dir
2016-05-17 21:52:42,035 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001/localmode-log-dir/history.txt.appattempt_1463521961992_0001_000000, maxErrors=10
2016-05-17 21:52:42,035 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:42,035 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521961992_0001_000000, appSubmitTime=1463521962019, launchTime=1463521962020
2016-05-17 21:52:42,040 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:42,042 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:42,043 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:42,043 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:42,044 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43199
2016-05-17 21:52:42,046 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:42,048 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521961992_0001_000000, startTime=1463521962048
2016-05-17 21:52:42,048 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:42,048 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:42,117 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:42,117 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:42,117 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463521961992_0001, dagName=[8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple
2016-05-17 21:52:42,131 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:42,131 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:42,132 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple
2016-05-17 21:52:42,134 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521961992_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001/localmode-log-dir/dag_1463521961992_0001_1.dot
2016-05-17 21:52:42,134 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001/localmode-log-dir/dag_1463521961992_0001_1-tez-dag.pb.txt
2016-05-17 21:52:42,147 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521961992_0001_1
2016-05-17 21:52:42,147 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,147 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple
2016-05-17 21:52:42 Running Dag: dag_1463521961992_0001_1
2016-05-17 21:52:42,163 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521961992_0001_1, submitTime=1463521962132
2016-05-17 21:52:42,163 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:42,164 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,164 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:42,165 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:42,165 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521961992_0001_1, initTime=1463521962163
2016-05-17 21:52:42,165 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961992_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:42,166 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_STARTED]: dagID=dag_1463521961992_0001_1, startTime=1463521962166
2016-05-17 21:52:42,166 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:42,166 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:42,166 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961992_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:42,166 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ED43F9BE1D8A4519B18FE2FDF9494E3D : {4E43EBC6419B4C5EAC4D309A32FA9BCA={InputName=4E43EBC6419B4C5EAC4D309A32FA9BCA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:42,167 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4E43EBC6419B4C5EAC4D309A32FA9BCA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:42,167 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,167 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463521961992_0001, dagName=[8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple
2016-05-17 21:52:42,167 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,167 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,167 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]. Starting root input initializers: 1
2016-05-17 21:52:42,168 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:42,168 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,168 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4E43EBC6419B4C5EAC4D309A32FA9BCA on vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,170 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:42,170 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4E43EBC6419B4C5EAC4D309A32FA9BCA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:42,171 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:42,171 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:42,171 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:42,171 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,172 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,172 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,172 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2194406ED73040119A986EB3879DF0C1, vertexId=vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:42,173 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:42,173 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2194406ED73040119A986EB3879DF0C1, vertexId=vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,175 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2194406ED73040119A986EB3879DF0C1, vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:42,175 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2194406ED73040119A986EB3879DF0C1, vertexId=vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,173 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4E43EBC6419B4C5EAC4D309A32FA9BCA on vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,177 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:42,177 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, vertexId=vertex_1463521961992_0001_1_01, initRequestedTime=1463521962168, initedTime=1463521962176, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:42,177 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:42,177 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:42,177 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4E43EBC6419B4C5EAC4D309A32FA9BCA=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:42,178 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] parallelism set to 1
2016-05-17 21:52:42,178 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:42,178 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521961992_0001_1_00, eventCount=1
2016-05-17 21:52:42,179 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:42,179 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, vertexId=vertex_1463521961992_0001_1_00, initRequestedTime=1463521962166, initedTime=1463521962179, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:42,180 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] with distanceFromRoot: 0
2016-05-17 21:52:42,180 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:42,180 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,180 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,181 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:42,181 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521961992_0001_1_00, startRequestedTime=1463521962180, startedTime=1463521962180
2016-05-17 21:52:42,181 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:42,181 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:42,182 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_STARTED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, taskId=task_1463521961992_0001_1_00_000000, scheduledTime=1463521962181, launchTime=1463521962181
2016-05-17 21:52:42,182 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:42,182 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521961992_0001_1_00 for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] numStartedSources: 1 numSources: 1
2016-05-17 21:52:42,182 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] with distanceFromRoot: 1
2016-05-17 21:52:42,182 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,182 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ED43F9BE1D8A4519B18FE2FDF9494E3D in vertex: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,183 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A6D55EBC661049FEA8EDA1BB694B72E1 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:42,183 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:42,183 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521961992_0001_1_01, startRequestedTime=1463521962182, startedTime=1463521962182
2016-05-17 21:52:42,183 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:42,184 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple, VertexName: ED43F9BE1D8A4519B18FE2FDF9494E3D, VertexParallelism: 1, TaskAttemptID:attempt_1463521961992_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4E43EBC6419B4C5EAC4D309A32FA9BCA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A6D55EBC661049FEA8EDA1BB694B72E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:42,184 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:42,184 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:42,184 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521961992_0001, with dag id: dag_1463521961992_0001_1
2016-05-17 21:52:42,185 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:42,185 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:42,185 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:42,186 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521961992_0001_00_000001
2016-05-17 21:52:42,186 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:42,188 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521961992_0001_00_000001
2016-05-17 21:52:42,188 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:42,189 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:42,190 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:42,190 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521961992_0001_00_000001, launchTime=1463521962190
2016-05-17 21:52:42,191 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961992_0001_00_000001
2016-05-17 21:52:42,193 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521961992_0001_1_00_000000_0] to container: [container_1463521961992_0001_00_000001]
2016-05-17 21:52:42,193 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:42,194 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521961992_0001_00_000001 given task: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,194 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961992_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42 Starting to run new task attempt: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,194 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521961992_0001_1_00_000000_0] started. Is using containerId: [container_1463521961992_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:42,194 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,195 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,195 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:42,195 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0, startTime=1463521962194, containerId=container_1463521961992_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521961992_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:42,195 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:42,196 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:42,196 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:42,196 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:42,196 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:42,196 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:42,196 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:42,197 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple, VertexName: ED43F9BE1D8A4519B18FE2FDF9494E3D, VertexParallelism: 1, TaskAttemptID:attempt_1463521961992_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4E43EBC6419B4C5EAC4D309A32FA9BCA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A6D55EBC661049FEA8EDA1BB694B72E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:42,197 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:42,197 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,197 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:42,198 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,198 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521961992_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:42,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4E43EBC6419B4C5EAC4D309A32FA9BCA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:42,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A6D55EBC661049FEA8EDA1BB694B72E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:42,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,201 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:42,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:42,203 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,203 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,206 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:42,206 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,208 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ED43F9BE1D8A4519B18FE2FDF9494E3D, ordinal: 0
2016-05-17 21:52:42,208 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ED43F9BE1D8A4519B18FE2FDF9494E3D, mem on start (mb), free: 474, total: 555, max: 672
2016-05-17 21:52:42,208 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,208 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:42,208 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:42,209 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,209 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,209 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:42,210 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:42,211 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,211 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,211 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:42,211 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:42,212 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:42,212 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:42,212 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:42,212 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:42,213 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:42,213 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4E43EBC6419B4C5EAC4D309A32FA9BCA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:42,213 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A6D55EBC661049FEA8EDA1BB694B72E1, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:42,213 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:42,213 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4E43EBC6419B4C5EAC4D309A32FA9BCA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:42,214 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:42,214 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:42,214 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,214 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:42,214 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,215 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:42,215 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,215 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:42,216 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5958b0ae
2016-05-17 21:52:42,217 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5958b0ae
2016-05-17 21:52:42,217 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:42,217 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:42,220 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:42,220 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,221 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: D334DFA40CE8414CBD0B9448AEA4B472
2016-05-17 21:52:42,221 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] D334DFA40CE8414CBD0B9448AEA4B472
2016-05-17 21:52:42,222 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@128be4e8; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1e6144db; comparator=cascading.tuple.hadoop.util.TupleComparator@3ef9572; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:42,222 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:42,223 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:42,223 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:42,236 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:42,237 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:42,237 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:42,237 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:42,237 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:42,238 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:42,238 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:42,238 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ED43F9BE1D8A4519B18FE2FDF9494E3D, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:42,241 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,241 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ED43F9BE1D8A4519B18FE2FDF9494E3D, mem on close (mb), free: 371, total: 555, max: 672
2016-05-17 21:52:42,241 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:42,241 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:42,241 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 21:52:42,241 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 21:52:42,245 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:42 Completed running task attempt: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,248 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:42,249 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521962239, Process_Duration=2, Process_End_Time=1463521962241, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 21:52:42,249 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521961992_0001_1_00, eventCount=1
2016-05-17 21:52:42,249 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:42,250 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0, startTime=1463521962194, finishTime=1463521962249, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962239, Process_Duration=2, Process_End_Time=1463521962241, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:42,250 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:42,250 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,250 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:42,250 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:42,250 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_FINISHED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, taskId=task_1463521961992_0001_1_00_000000, startTime=1463521962194, finishTime=1463521962250, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463521961992_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962239, Process_Duration=2, Process_End_Time=1463521962241, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:42,251 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:42,251 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] : 1
2016-05-17 21:52:42,251 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,251 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:42,251 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,252 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961992_0001_00_000001
2016-05-17 21:52:42,252 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:42,252 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_FINISHED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, vertexId=vertex_1463521961992_0001_1_00, initRequestedTime=1463521962166, initedTime=1463521962179, startRequestedTime=1463521962180, startedTime=1463521962180, finishTime=1463521962251, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962239, Process_Duration=2, Process_End_Time=1463521962241, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463521962194, firstTasksToStart=[ task_1463521961992_0001_1_00_000000 ], lastTaskFinishTime=1463521962249, lastTasksToFinish=[ task_1463521961992_0001_1_00_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521961992_0001_1_00_000000 ], longestDurationTasks=[ task_1463521961992_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:42,252 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:42,252 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:42,253 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] attempt: attempt_1463521961992_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:42,253 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,253 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,253 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521961992_0001_00_000001, stoppedTime=1463521962253, exitStatus=0
2016-05-17 21:52:42,253 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:42,253 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:42,253 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:42,254 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:42,253 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_STARTED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, taskId=task_1463521961992_0001_1_01_000000, scheduledTime=1463521962253, launchTime=1463521962253
2016-05-17 21:52:42,254 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:42,255 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,255 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521961992_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:42,255 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple, VertexName: A6D55EBC661049FEA8EDA1BB694B72E1, VertexParallelism: 1, TaskAttemptID:attempt_1463521961992_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2194406ED73040119A986EB3879DF0C1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:42,255 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:42,255 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:42,255 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:42,257 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521961992_0001_00_000002
2016-05-17 21:52:42,257 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:42,259 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463521961992_0001_00_000002
2016-05-17 21:52:42,259 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:42,260 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:42,260 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:42,260 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521961992_0001_00_000002, launchTime=1463521962259
2016-05-17 21:52:42,260 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961992_0001_00_000002
2016-05-17 21:52:42,261 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521961992_0001_1_01_000000_0] to container: [container_1463521961992_0001_00_000002]
2016-05-17 21:52:42,261 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:42,262 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521961992_0001_1_01_000000_0] started. Is using containerId: [container_1463521961992_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:42,262 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0, startTime=1463521962262, containerId=container_1463521961992_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521961992_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:42,262 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:42,262 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521961992_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:42,262 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961992_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:42,262 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:42,263 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521961992_0001_00_000001, Exiting
2016-05-17 21:52:42,263 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:42,263 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521961992_0001_00_000002 given task: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,263 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521961992_0001_00_000001
2016-05-17 21:52:42,263 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521961992_0001_00_000001
2016-05-17 21:52:42,264 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961992_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42 Starting to run new task attempt: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,264 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,264 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,265 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:42,265 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:42,265 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:42,265 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:42,265 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:42,265 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple, VertexName: A6D55EBC661049FEA8EDA1BB694B72E1, VertexParallelism: 1, TaskAttemptID:attempt_1463521961992_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2194406ED73040119A986EB3879DF0C1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:42,265 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521961992_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:42,265 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:42,265 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521961992_0001_00_000001 completed successfully
2016-05-17 21:52:42,266 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:42,266 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,266 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:42,266 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,267 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521961992_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:42,267 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:42,267 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:42,268 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,268 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2194406ED73040119A986EB3879DF0C1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:42,267 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,270 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:42,270 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:42,270 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,271 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,271 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:42,272 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:42,273 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2194406ED73040119A986EB3879DF0C1
2016-05-17 21:52:42,273 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:42,274 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2194406ED73040119A986EB3879DF0C1
2016-05-17 21:52:42,276 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A6D55EBC661049FEA8EDA1BB694B72E1, ordinal: 1
2016-05-17 21:52:42,276 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6D55EBC661049FEA8EDA1BB694B72E1, mem on start (mb), free: 367, total: 555, max: 672
2016-05-17 21:52:42,276 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,276 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:42,276 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:42,283 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:42,283 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2194406ED73040119A986EB3879DF0C1
2016-05-17 21:52:42,283 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:42,283 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:42,283 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:42,283 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:42,284 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:42,284 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:42,284 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:42,284 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ED43F9BE1D8A4519B18FE2FDF9494E3D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:42,284 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2194406ED73040119A986EB3879DF0C1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:42,284 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:42,284 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ED43F9BE1D8A4519B18FE2FDF9494E3D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:42,284 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:42,284 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:42,284 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,287 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:42,288 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:42,288 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ED43F9BE1D8A4519B18FE2FDF9494E3D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:42,289 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:42,289 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:42,289 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:42,290 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,291 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:42,292 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,293 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521961992_0001_1_00_000000_0_10089, runDuration: 39458, ]
2016-05-17 21:52:42,295 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521961992_0001_1_00_000000_0_10089] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463521962295, TimeTaken=2, Rate=0.10 MB/s
2016-05-17 21:52:42,295 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,295 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:42,295 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [ED43F9BE1D8A4519B18FE2FDF9494E3D] #14 in 2ms
2016-05-17 21:52:42,296 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ED43F9BE1D8A4519B18FE2FDF9494E3D] #14, status:false, isInterrupted:false
2016-05-17 21:52:42,296 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:42,297 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:42,298 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: D334DFA40CE8414CBD0B9448AEA4B472
2016-05-17 21:52:42,298 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"], id: 2194406ED73040119A986EB3879DF0C1
2016-05-17 21:52:42,298 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 21:52:42,298 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:42,298 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:42,299 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 21:52:42,299 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,299 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:42,298 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-17 21:52:42,300 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] D334DFA40CE8414CBD0B9448AEA4B472, for 1 inputs
2016-05-17 21:52:42,300 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A6D55EBC661049FEA8EDA1BB694B72E1, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:42,302 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@9d68d7a; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@64e249a1
2016-05-17 21:52:42,305 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:42,306 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:42,307 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,307 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521961992_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:42,308 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,308 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6D55EBC661049FEA8EDA1BB694B72E1, mem on close (mb), free: 363, total: 555, max: 672
2016-05-17 21:52:42,308 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,309 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:42,309 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:42,310 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:42,310 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:42,310 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:42 Completed running task attempt: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,312 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:42,313 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521962300, Process_Duration=8, Process_End_Time=1463521962308, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 21:52:42,313 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0, startTime=1463521962262, finishTime=1463521962313, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962300, Process_Duration=8, Process_End_Time=1463521962308, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:42,313 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:42,314 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 21:52:42,314 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,313 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:42,314 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961992_0001_00_000002
2016-05-17 21:52:42,314 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,315 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:42,315 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_FINISHED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, taskId=task_1463521961992_0001_1_01_000000, startTime=1463521962262, finishTime=1463521962315, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463521961992_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962300, Process_Duration=8, Process_End_Time=1463521962308, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:42,315 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:42,315 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] : 1
2016-05-17 21:52:42,316 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,316 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:42,316 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:42,317 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_FINISHED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, vertexId=vertex_1463521961992_0001_1_01, initRequestedTime=1463521962168, initedTime=1463521962176, startRequestedTime=1463521962182, startedTime=1463521962182, finishTime=1463521962316, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962300, Process_Duration=8, Process_End_Time=1463521962308, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463521962262, firstTasksToStart=[ task_1463521961992_0001_1_01_000000 ], lastTaskFinishTime=1463521962313, lastTasksToFinish=[ task_1463521961992_0001_1_01_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521961992_0001_1_01_000000 ], longestDurationTasks=[ task_1463521961992_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:42,317 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:42,318 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:42,318 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521961992_0001_00_000002, stoppedTime=1463521962318, exitStatus=0
2016-05-17 21:52:42,318 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:42,318 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:42,319 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:42,319 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521961992_0001_1
2016-05-17 21:52:42,319 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521961992_0001_1
2016-05-17 21:52:42,320 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2194406ED73040119A986EB3879DF0C1 for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,325 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521961992_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:42,325 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961992_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:42,326 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521961992_0001_00_000002, Exiting
2016-05-17 21:52:42,326 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521961992_0001_00_000002
2016-05-17 21:52:42,326 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521961992_0001_00_000002
2016-05-17 21:52:42,326 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521961992_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:42,326 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521961992_0001_00_000002 completed successfully
2016-05-17 21:52:42,327 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,329 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521961992_0001_1, queueSize=0
2016-05-17 21:52:42,329 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:42,329 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521961992_0001_1, startTime=1463521962166, finishTime=1463521962327, timeTaken=161, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1163919360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043924539, Process_Duration=10, Process_End_Time=2927043924549, Read_Duration=0, Tuples_Read=20, Tuples_Written=18, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 21:52:42,329 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521961992_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:42,329 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961992_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:42,330 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521961992_0001_00_000002
2016-05-17 21:52:42,330 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:42 Completed Dag: dag_1463521961992_0001_1
2016-05-17 21:52:42,333 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521961992_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:42,334 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521961992_0001_1_post
2016-05-17 21:52:42,334 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,334 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:42,336 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463521961992_0001
2016-05-17 21:52:42,338 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:42,338 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:42,338 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:42,338 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:42,338 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:42,338 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:42,340 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:42,340 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:42,340 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:42,340 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:42,340 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:42,341 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:42,341 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:42,346 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001
2016-05-17 21:52:42,346 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple/_temporary
2016-05-17 21:52:42,347 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.373
2016-05-17 21:52:42,382 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimple/simple

Standard error

2016-05-17 21:52:30 Running Dag: dag_1463521949609_0001_1
2016-05-17 21:52:30 Starting to run new task attempt: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31 Completed running task attempt: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31 Starting to run new task attempt: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31 Completed running task attempt: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31 Starting to run new task attempt: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31 Completed running task attempt: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31 Completed Dag: dag_1463521949609_0001_1
2016-05-17 21:52:32 Running Dag: dag_1463521952250_0001_1
2016-05-17 21:52:32 Starting to run new task attempt: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32 Completed running task attempt: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32 Starting to run new task attempt: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32 Completed running task attempt: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32 Completed Dag: dag_1463521952250_0001_1
2016-05-17 21:52:33 Running Dag: dag_1463521953182_0001_1
2016-05-17 21:52:33 Starting to run new task attempt: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33 Completed running task attempt: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33 Completed Dag: dag_1463521953182_0001_1
2016-05-17 21:52:33 Running Dag: dag_1463521953701_0001_1
2016-05-17 21:52:33 Starting to run new task attempt: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33 Completed running task attempt: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33 Completed Dag: dag_1463521953701_0001_1
2016-05-17 21:52:34 Running Dag: dag_1463521954386_0001_1
2016-05-17 21:52:34 Starting to run new task attempt: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34 Starting to run new task attempt: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34 Completed running task attempt: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34 Completed running task attempt: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34 Starting to run new task attempt: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35 Completed running task attempt: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35 Completed Dag: dag_1463521954386_0001_1
2016-05-17 21:52:35 Running Dag: dag_1463521955407_0001_1
2016-05-17 21:52:35 Starting to run new task attempt: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35 Completed running task attempt: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35 Starting to run new task attempt: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35 Completed running task attempt: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35 Completed Dag: dag_1463521955407_0001_1
2016-05-17 21:52:36 Running Dag: dag_1463521956197_0001_1
2016-05-17 21:52:36 Starting to run new task attempt: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36 Completed running task attempt: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36 Starting to run new task attempt: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36 Completed running task attempt: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36 Completed Dag: dag_1463521956197_0001_1
2016-05-17 21:52:37 Running Dag: dag_1463521956861_0001_1
2016-05-17 21:52:37 Starting to run new task attempt: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37 Completed running task attempt: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37 Completed Dag: dag_1463521956861_0001_1
2016-05-17 21:52:37 Running Dag: dag_1463521957355_0001_1
2016-05-17 21:52:37 Starting to run new task attempt: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37 Completed running task attempt: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37 Starting to run new task attempt: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37 Completed running task attempt: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37 Completed Dag: dag_1463521957355_0001_1
2016-05-17 21:52:38 Running Dag: dag_1463521958105_0001_1
2016-05-17 21:52:38 Starting to run new task attempt: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38 Starting to run new task attempt: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38 Completed running task attempt: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38 Completed running task attempt: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38 Starting to run new task attempt: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39 Completed running task attempt: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39 Completed Dag: dag_1463521958105_0001_1
2016-05-17 21:52:39 Running Dag: dag_1463521959303_0001_1
2016-05-17 21:52:39 Starting to run new task attempt: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39 Completed running task attempt: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39 Completed Dag: dag_1463521959303_0001_1
2016-05-17 21:52:39 Running Dag: dag_1463521959775_0001_1
2016-05-17 21:52:39 Starting to run new task attempt: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40 Starting to run new task attempt: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40 Completed Dag: dag_1463521959775_0001_1
2016-05-17 21:52:40 Running Dag: dag_1463521960395_0001_1
2016-05-17 21:52:40 Starting to run new task attempt: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40 Starting to run new task attempt: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40 Completed Dag: dag_1463521960395_0001_1
2016-05-17 21:52:41 Running Dag: dag_1463521960977_0001_1
2016-05-17 21:52:41 Starting to run new task attempt: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41 Completed running task attempt: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41 Starting to run new task attempt: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41 Completed running task attempt: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41 Completed Dag: dag_1463521960977_0001_1
2016-05-17 21:52:41 Running Dag: dag_1463521961509_0001_1
2016-05-17 21:52:41 Starting to run new task attempt: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41 Completed running task attempt: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41 Completed Dag: dag_1463521961509_0001_1
2016-05-17 21:52:42 Running Dag: dag_1463521961992_0001_1
2016-05-17 21:52:42 Starting to run new task attempt: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42 Completed running task attempt: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42 Starting to run new task attempt: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42 Completed running task attempt: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42 Completed Dag: dag_1463521961992_0001_1