2016-05-17 21:52:27,423 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-17 21:52:27,454 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.BasicPipesPlatformTest, with platform: hadoop2-tez
2016-05-17 21:52:27,490 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-17 21:52:27,850 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-17 21:52:28,206 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-17 21:52:28,426 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:28,429 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:28,544 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: A74499D90A2B4DBF98BC2F2E01DD4D17
2016-05-17 21:52:28,738 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.089
2016-05-17 21:52:28,773 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.125
2016-05-17 21:52:28,774 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:28,775 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:28,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:28,926 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:29,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-17 21:52:29,397 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-17 21:52:29,400 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:29,401 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:29,402 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-17 21:52:29,403 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:29,403 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:29,403 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:29,405 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:29,405 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:29,428 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:29,602 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:29,610 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:29,610 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:29,650 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001 doesn't exist and is created
2016-05-17 21:52:29,711 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:29,711 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:29,713 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001
2016-05-17 21:52:29,758 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521949609_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:29,810 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:29,812 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-17 21:52:29,812 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-17 21:52:29,812 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:29,852 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:29,884 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:29,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:29,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001/localmode-log-dir
2016-05-17 21:52:29,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001/localmode-log-dir/history.txt.appattempt_1463521949609_0001_000000, maxErrors=10
2016-05-17 21:52:29,893 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:29,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521949609_0001_000000, appSubmitTime=1463521949738, launchTime=1463521949757
2016-05-17 21:52:29,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:29,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:29,902 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:29,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:29,913 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 21:52:29,913 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 21:52:29,913 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:29,960 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47089
2016-05-17 21:52:29,999 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:30,013 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 21:52:30,014 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 21:52:30,014 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:30,071 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:30,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521949609_0001_000000, startTime=1463521950071
2016-05-17 21:52:30,072 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:30,114 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:30,130 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:30,130 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463521949609_0001, dagName=[231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:30,335 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:30,432 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521949609_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001/localmode-log-dir/dag_1463521949609_0001_1.dot
2016-05-17 21:52:30,437 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001/localmode-log-dir/dag_1463521949609_0001_1-tez-dag.pb.txt
2016-05-17 21:52:30,534 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521949609_0001_1
2016-05-17 21:52:30,535 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:30,535 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:30 Running Dag: dag_1463521949609_0001_1
2016-05-17 21:52:30,564 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:30,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521949609_0001_1, submitTime=1463521950335
2016-05-17 21:52:30,593 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:30,600 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:30,602 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:30,602 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521949609_0001_1, initTime=1463521950565
2016-05-17 21:52:30,603 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521949609_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:30,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:30,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_STARTED]: dagID=dag_1463521949609_0001_1, startTime=1463521950604
2016-05-17 21:52:30,605 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:30,607 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521949609_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:30,608 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BC12AAA767694F8E8C505DF617C89107 : {3C2FF4BCEB3745B0AE2241B89B16E362={InputName=3C2FF4BCEB3745B0AE2241B89B16E362}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:30,610 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3C2FF4BCEB3745B0AE2241B89B16E362, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:30,610 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,615 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,615 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,616 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]. Starting root input initializers: 1
2016-05-17 21:52:30,617 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463521949609_0001, dagName=[231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 21:52:30,631 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3C2FF4BCEB3745B0AE2241B89B16E362 on vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,633 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:30,636 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:30,695 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521949609_0001, with dag id: dag_1463521949609_0001_1
2016-05-17 21:52:30,700 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:30,700 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:30,700 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:30,703 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3C2FF4BCEB3745B0AE2241B89B16E362 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:30,704 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:30,705 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:30,719 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:30,722 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:30,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, vertexId=vertex_1463521949609_0001_1_01, initRequestedTime=1463521950634, initedTime=1463521950721, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:30,723 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:30,731 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,755 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:30,756 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,763 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,763 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,764 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=27DB08F568EF4E09B6365C456C48AFD3, vertexId=vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:30,767 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:30,769 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3C2FF4BCEB3745B0AE2241B89B16E362 on vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,772 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=27DB08F568EF4E09B6365C456C48AFD3, vertexId=vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,790 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=27DB08F568EF4E09B6365C456C48AFD3, vertexName=E959E740B0D74FE8951993F5CB9EF4D4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:30,801 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=27DB08F568EF4E09B6365C456C48AFD3, vertexId=vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:30,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:30,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, vertexId=vertex_1463521949609_0001_1_02, initRequestedTime=1463521950731, initedTime=1463521950805, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:30,805 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:30,810 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:30,810 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3C2FF4BCEB3745B0AE2241B89B16E362=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:30,811 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] parallelism set to 1
2016-05-17 21:52:30,811 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:30,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521949609_0001_1_00, eventCount=1
2016-05-17 21:52:30,822 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:30,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BC12AAA767694F8E8C505DF617C89107, vertexId=vertex_1463521949609_0001_1_00, initRequestedTime=1463521950608, initedTime=1463521950822, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:30,823 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] with distanceFromRoot: 0
2016-05-17 21:52:30,823 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:30,832 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:30,833 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:30,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521949609_0001_1_00, startRequestedTime=1463521950831, startedTime=1463521950831
2016-05-17 21:52:30,837 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:30,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:30,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_STARTED]: vertexName=BC12AAA767694F8E8C505DF617C89107, taskId=task_1463521949609_0001_1_00_000000, scheduledTime=1463521950864, launchTime=1463521950864
2016-05-17 21:52:30,865 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:30,866 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521949609_0001_1_00 for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] numStartedSources: 1 numSources: 1
2016-05-17 21:52:30,866 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] with distanceFromRoot: 1
2016-05-17 21:52:30,868 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:30,872 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BC12AAA767694F8E8C505DF617C89107 in vertex: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:30,874 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BD9CBF0804B84E23A67381209AA8AA79 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:30,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:30,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521949609_0001_1_01, startRequestedTime=1463521950868, startedTime=1463521950868
2016-05-17 21:52:30,875 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:30,877 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: BC12AAA767694F8E8C505DF617C89107, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3C2FF4BCEB3745B0AE2241B89B16E362, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:30,879 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:30,879 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521949609_0001_1_01 for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] numStartedSources: 1 numSources: 1
2016-05-17 21:52:30,880 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] with distanceFromRoot: 2
2016-05-17 21:52:30,880 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:30,883 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BD9CBF0804B84E23A67381209AA8AA79 in vertex: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:30,883 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E959E740B0D74FE8951993F5CB9EF4D4 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:30,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521949609_0001_1_02, startRequestedTime=1463521950883, startedTime=1463521950883
2016-05-17 21:52:30,884 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:30,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:30,925 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:30,930 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:30,930 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:30,934 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521949609_0001_00_000001
2016-05-17 21:52:30,934 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:30,942 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521949609_0001_00_000001
2016-05-17 21:52:30,942 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:30,947 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:30,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:30,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521949609_0001_00_000001, launchTime=1463521950947
2016-05-17 21:52:30,951 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000001
2016-05-17 21:52:30,956 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521949609_0001_1_00_000000_0] to container: [container_1463521949609_0001_00_000001]
2016-05-17 21:52:30,956 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:30,958 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521949609_0001_00_000001 given task: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:30,958 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:30 Starting to run new task attempt: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:30,959 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:30,960 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:30,960 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521949609_0001_1_00_000000_0] started. Is using containerId: [container_1463521949609_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:30,961 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:30,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:30,962 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:30,962 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:30,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BC12AAA767694F8E8C505DF617C89107, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0, startTime=1463521950958, containerId=container_1463521949609_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521949609_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:30,964 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:30,964 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-17 21:52:30,965 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:30,968 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:30,989 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:30,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: BC12AAA767694F8E8C505DF617C89107, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3C2FF4BCEB3745B0AE2241B89B16E362, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:30,993 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:31,000 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:31,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,004 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521949609_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:31,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3C2FF4BCEB3745B0AE2241B89B16E362, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:31,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:31,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:31,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:31,017 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,037 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:31,038 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:31,039 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,043 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:31,043 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,053 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BC12AAA767694F8E8C505DF617C89107, ordinal: 0
2016-05-17 21:52:31,054 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC12AAA767694F8E8C505DF617C89107, mem on start (mb), free: 236, total: 281, max: 672
2016-05-17 21:52:31,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:31,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:31,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:31,057 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:31,061 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:31,062 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:31,062 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:31,063 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:31,063 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:31,064 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3C2FF4BCEB3745B0AE2241B89B16E362, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:31,064 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BD9CBF0804B84E23A67381209AA8AA79, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:31,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:31,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3C2FF4BCEB3745B0AE2241B89B16E362 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:31,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:31,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:31,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,068 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:31,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:31,071 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,072 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:31,083 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@49c9ccdb
2016-05-17 21:52:31,094 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@49c9ccdb
2016-05-17 21:52:31,095 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:31,096 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:31,111 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:31,153 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 3C2FF4BCEB3745B0AE2241B89B16E362
2016-05-17 21:52:31,153 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: D095011A42D942A78C13526AE8171DF1
2016-05-17 21:52:31,154 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] D095011A42D942A78C13526AE8171DF1
2016-05-17 21:52:31,200 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6742eba7; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@5a7d55e7; comparator=cascading.tuple.hadoop.util.TupleComparator@746ab4a7; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:31,205 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:31,208 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:31,209 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:31,334 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:31,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:31,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:31,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:31,335 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:31,337 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:31,337 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:31,339 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BC12AAA767694F8E8C505DF617C89107, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:52:31,348 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,348 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC12AAA767694F8E8C505DF617C89107, mem on close (mb), free: 123, total: 281, max: 672
2016-05-17 21:52:31,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:31,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:31,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 21:52:31,350 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 21:52:31,366 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:31 Completed running task attempt: attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,389 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:31,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521949609_0001_1_00, eventCount=1
2016-05-17 21:52:31,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521951340, Process_Duration=8, Process_End_Time=1463521951348, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 21:52:31,393 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:31,393 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:31,394 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,394 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000001
2016-05-17 21:52:31,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BC12AAA767694F8E8C505DF617C89107, taskAttemptId=attempt_1463521949609_0001_1_00_000000_0, startTime=1463521950958, finishTime=1463521951390, timeTaken=432, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951340, Process_Duration=8, Process_End_Time=1463521951348, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:31,398 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:31,398 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:31,402 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521949609_0001_1_00_000000_0
2016-05-17 21:52:31,404 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:31,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_FINISHED]: vertexName=BC12AAA767694F8E8C505DF617C89107, taskId=task_1463521949609_0001_1_00_000000, startTime=1463521950958, finishTime=1463521951403, timeTaken=445, status=SUCCEEDED, successfulAttemptID=attempt_1463521949609_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951340, Process_Duration=8, Process_End_Time=1463521951348, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:31,409 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:31,411 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:31,412 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] : 1
2016-05-17 21:52:31,412 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:31,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:31,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_FINISHED]: vertexName=BC12AAA767694F8E8C505DF617C89107, vertexId=vertex_1463521949609_0001_1_00, initRequestedTime=1463521950608, initedTime=1463521950822, startRequestedTime=1463521950831, startedTime=1463521950831, finishTime=1463521951412, timeTaken=581, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951340, Process_Duration=8, Process_End_Time=1463521951348, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463521950958, firstTasksToStart=[ task_1463521949609_0001_1_00_000000 ], lastTaskFinishTime=1463521951390, lastTasksToFinish=[ task_1463521949609_0001_1_00_000000 ], minTaskDuration=432, maxTaskDuration=432, avgTaskDuration=432.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521949609_0001_1_00_000000 ], longestDurationTasks=[ task_1463521949609_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:31,427 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:31,428 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521949609_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:31,429 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000001: 34 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:31,429 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521949609_0001_00_000001, Exiting
2016-05-17 21:52:31,430 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521949609_0001_00_000001
2016-05-17 21:52:31,430 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521949609_0001_00_000001
2016-05-17 21:52:31,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:31,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521949609_0001_00_000001, stoppedTime=1463521951431, exitStatus=0
2016-05-17 21:52:31,433 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:31,434 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] attempt: attempt_1463521949609_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:31,435 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521949609_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:31,436 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521949609_0001_00_000001 completed successfully
2016-05-17 21:52:31,436 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,438 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:31,441 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:31,441 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:31,441 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521949609_0001_00_000001
2016-05-17 21:52:31,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:31,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_STARTED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, taskId=task_1463521949609_0001_1_01_000000, scheduledTime=1463521951443, launchTime=1463521951443
2016-05-17 21:52:31,443 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:31,445 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:31,445 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: BD9CBF0804B84E23A67381209AA8AA79, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BC12AAA767694F8E8C505DF617C89107, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E959E740B0D74FE8951993F5CB9EF4D4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:31,446 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:31,446 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:31,447 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521949609_0001_00_000002
2016-05-17 21:52:31,447 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:31,452 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521949609_0001_00_000002
2016-05-17 21:52:31,453 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:31,456 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:31,457 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:31,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521949609_0001_00_000002, launchTime=1463521951456
2016-05-17 21:52:31,459 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000002
2016-05-17 21:52:31,460 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521949609_0001_1_01_000000_0] to container: [container_1463521949609_0001_00_000002]
2016-05-17 21:52:31,461 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:31,461 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521949609_0001_1_01_000000_0] started. Is using containerId: [container_1463521949609_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:31,462 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:31,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0, startTime=1463521951461, containerId=container_1463521949609_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521949609_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:31,462 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:31,462 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:31,463 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521949609_0001_00_000002 given task: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,463 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31 Starting to run new task attempt: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,464 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,464 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,465 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:31,465 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:31,465 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:31,465 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:31,466 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:31,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: BD9CBF0804B84E23A67381209AA8AA79, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BC12AAA767694F8E8C505DF617C89107, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E959E740B0D74FE8951993F5CB9EF4D4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:31,467 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:31,467 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:31,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BC12AAA767694F8E8C505DF617C89107, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:31,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:31,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E959E740B0D74FE8951993F5CB9EF4D4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:31,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:31,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:31,480 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:31,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,485 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:31,486 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:31,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:31,496 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BD9CBF0804B84E23A67381209AA8AA79, ordinal: 1
2016-05-17 21:52:31,496 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD9CBF0804B84E23A67381209AA8AA79, mem on start (mb), free: 113, total: 281, max: 672
2016-05-17 21:52:31,496 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,496 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:31,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:31,501 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521949609_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:31,507 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:31,507 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:31,507 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:31,510 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:31,510 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:31,512 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:31,512 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:31,512 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 739036352, TotalRequestedScaled: 3.69518176E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:1.05
2016-05-17 21:52:31,513 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E959E740B0D74FE8951993F5CB9EF4D4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=66985083
2016-05-17 21:52:31,513 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BC12AAA767694F8E8C505DF617C89107, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=405125774
2016-05-17 21:52:31,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:31,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BC12AAA767694F8E8C505DF617C89107 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:31,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:31,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:31,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,525 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:31,527 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:31,531 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BC12AAA767694F8E8C505DF617C89107 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:31,534 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=405125774. Updated to: ShuffleMem=405125774, postMergeMem: 0
2016-05-17 21:52:31,535 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=405125774, maxSingleShuffleLimit=101281440, mergeThreshold=364613184, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:31,538 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:31,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:31,544 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,548 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521949609_0001_1_00_000000_0_10003, runDuration: 343922, ]
2016-05-17 21:52:31,550 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:31,558 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: D095011A42D942A78C13526AE8171DF1
2016-05-17 21:52:31,558 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: D0EC84A6300E4617A6DFD1D38B5DC7A4
2016-05-17 21:52:31,559 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521949609_0001_1_00_000000_0_10003] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463521951559, TimeTaken=5, Rate=0.04 MB/s
2016-05-17 21:52:31,559 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,560 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:31,560 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BC12AAA767694F8E8C505DF617C89107] #1 in 7ms
2016-05-17 21:52:31,560 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] D0EC84A6300E4617A6DFD1D38B5DC7A4
2016-05-17 21:52:31,561 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BC12AAA767694F8E8C505DF617C89107] #1, status:false, isInterrupted:false
2016-05-17 21:52:31,565 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:31,568 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 21:52:31,569 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:31,570 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@58f03ef7; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@4b5f7bf; comparator=cascading.tuple.hadoop.util.TupleComparator@6841c69c; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:31,570 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:31,571 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:31,572 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:31,574 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:31,642 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 16515068(66060272)
2016-05-17 21:52:31,642 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 63
2016-05-17 21:52:31,643 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 52848232
2016-05-17 21:52:31,644 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 66060288
2016-05-17 21:52:31,644 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 16515068; length = 4128768
2016-05-17 21:52:31,646 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] D095011A42D942A78C13526AE8171DF1, for 1 inputs
2016-05-17 21:52:31,648 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 21:52:31,648 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,648 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BD9CBF0804B84E23A67381209AA8AA79, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:52:31,648 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:31,653 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5e73a225; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@799a5b94
2016-05-17 21:52:31,662 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,662 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD9CBF0804B84E23A67381209AA8AA79, mem on close (mb), free: 105, total: 345, max: 672
2016-05-17 21:52:31,662 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BC12AAA767694F8E8C505DF617C89107
2016-05-17 21:52:31,664 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:31,665 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:31,665 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:31,666 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:31,666 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:31,667 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:31,667 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:31,667 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 184; bufvoid = 66060288
2016-05-17 21:52:31,668 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 16515068(66060272); kvend = 16515040(66060160); length = 29/4128768
2016-05-17 21:52:31,671 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:31 Completed running task attempt: attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,676 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:31,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521949609_0001_1_01, eventCount=1
2016-05-17 21:52:31,678 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:31,680 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 49 [[File System Counters FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=121, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521951648, Process_Duration=14, Process_End_Time=1463521951662, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=1]]
2016-05-17 21:52:31,681 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:31,681 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, taskAttemptId=attempt_1463521949609_0001_1_01_000000_0, startTime=1463521951461, finishTime=1463521951678, timeTaken=217, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=121, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951648, Process_Duration=14, Process_End_Time=1463521951662, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=1
2016-05-17 21:52:31,682 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:31,683 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521949609_0001_1_01_000000_0
2016-05-17 21:52:31,683 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:31,682 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000002
2016-05-17 21:52:31,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_FINISHED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, taskId=task_1463521949609_0001_1_01_000000, startTime=1463521951461, finishTime=1463521951683, timeTaken=222, status=SUCCEEDED, successfulAttemptID=attempt_1463521949609_0001_1_01_000000_0, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=121, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951648, Process_Duration=14, Process_End_Time=1463521951662, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=1
2016-05-17 21:52:31,686 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:31,687 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] : 1
2016-05-17 21:52:31,687 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:31,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:31,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:31,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_FINISHED]: vertexName=BD9CBF0804B84E23A67381209AA8AA79, vertexId=vertex_1463521949609_0001_1_01, initRequestedTime=1463521950634, initedTime=1463521950721, startRequestedTime=1463521950868, startedTime=1463521950868, finishTime=1463521951688, timeTaken=820, status=SUCCEEDED, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=121, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951648, Process_Duration=14, Process_End_Time=1463521951662, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=1, vertexStats=firstTaskStartTime=1463521951461, firstTasksToStart=[ task_1463521949609_0001_1_01_000000 ], lastTaskFinishTime=1463521951678, lastTasksToFinish=[ task_1463521949609_0001_1_01_000000 ], minTaskDuration=217, maxTaskDuration=217, avgTaskDuration=217.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521949609_0001_1_01_000000 ], longestDurationTasks=[ task_1463521949609_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:31,695 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:31,696 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:31,696 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:31,696 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521949609_0001_00_000002, stoppedTime=1463521951696, exitStatus=0
2016-05-17 21:52:31,697 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:31,697 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] attempt: attempt_1463521949609_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:31,697 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E959E740B0D74FE8951993F5CB9EF4D4
2016-05-17 21:52:31,698 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:31,698 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:31,698 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521949609_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:52:31,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_STARTED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, taskId=task_1463521949609_0001_1_02_000000, scheduledTime=1463521951698, launchTime=1463521951698
2016-05-17 21:52:31,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:31,699 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:31,699 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:31,700 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:31,700 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: E959E740B0D74FE8951993F5CB9EF4D4, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=27DB08F568EF4E09B6365C456C48AFD3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:31,700 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:31,700 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:31,701 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521949609_0001_00_000003
2016-05-17 21:52:31,701 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:31,703 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521949609_0001_00_000003
2016-05-17 21:52:31,703 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - TezTask metrics system already initialized!
2016-05-17 21:52:31,704 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:31,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521949609_0001_00_000003, launchTime=1463521951704
2016-05-17 21:52:31,704 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521949609_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:31,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:31,705 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000003
2016-05-17 21:52:31,705 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:31,705 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521949609_0001_00_000002, Exiting
2016-05-17 21:52:31,705 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521949609_0001_00_000002
2016-05-17 21:52:31,705 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521949609_0001_00_000002
2016-05-17 21:52:31,706 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521949609_0001_1_02_000000_0] to container: [container_1463521949609_0001_00_000003]
2016-05-17 21:52:31,707 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521949609_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:31,707 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:31,707 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521949609_0001_00_000002 completed successfully
2016-05-17 21:52:31,708 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:31,708 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521949609_0001_00_000003 given task: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,709 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521949609_0001_1_02_000000_0] started. Is using containerId: [container_1463521949609_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 21:52:31,709 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:31 Starting to run new task attempt: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,710 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,710 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,711 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:31,711 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:31,711 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:31,711 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:31,711 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:31,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4] test/(1/1) ...stSimpleChain/simplechain, VertexName: E959E740B0D74FE8951993F5CB9EF4D4, VertexParallelism: 1, TaskAttemptID:attempt_1463521949609_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=27DB08F568EF4E09B6365C456C48AFD3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:31,711 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:31,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0, startTime=1463521951709, containerId=container_1463521949609_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521949609_0001_00_000003/teamcity, completedLogs=
2016-05-17 21:52:31,712 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:31,713 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:31,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,713 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521949609_0001_1_02_000000_0 sent events: (0-1)
2016-05-17 21:52:31,713 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:31,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BD9CBF0804B84E23A67381209AA8AA79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:31,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=27DB08F568EF4E09B6365C456C48AFD3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:31,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:31,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:31,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 27DB08F568EF4E09B6365C456C48AFD3
2016-05-17 21:52:31,715 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:31,715 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 27DB08F568EF4E09B6365C456C48AFD3
2016-05-17 21:52:31,716 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:31,716 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:31,716 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,727 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:31,741 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:31,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 27DB08F568EF4E09B6365C456C48AFD3
2016-05-17 21:52:31,744 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E959E740B0D74FE8951993F5CB9EF4D4, ordinal: 2
2016-05-17 21:52:31,744 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E959E740B0D74FE8951993F5CB9EF4D4, mem on start (mb), free: 98, total: 345, max: 672
2016-05-17 21:52:31,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:31,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:31,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:31,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:31,745 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:31,745 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:31,745 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:31,746 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:31,746 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:31,746 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:31,747 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 27DB08F568EF4E09B6365C456C48AFD3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:31,747 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BD9CBF0804B84E23A67381209AA8AA79, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:31,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:31,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BD9CBF0804B84E23A67381209AA8AA79 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:31,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:31,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:31,749 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:31,750 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:31,750 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BD9CBF0804B84E23A67381209AA8AA79 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:31,750 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:31,750 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:31,751 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:31,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:31,754 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,754 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521949609_0001_1_01_000000_0_10006, runDuration: 198687, ]
2016-05-17 21:52:31,759 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521949609_0001_1_01_000000_0_10006] to DISK_DIRECT, CompressedSize=210, DecompressedSize=206,EndTime=1463521951759, TimeTaken=2, Rate=0.10 MB/s
2016-05-17 21:52:31,759 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,759 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:31,759 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:31,760 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BD9CBF0804B84E23A67381209AA8AA79] #2 in 5ms
2016-05-17 21:52:31,761 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BD9CBF0804B84E23A67381209AA8AA79] #2, status:false, isInterrupted:false
2016-05-17 21:52:31,762 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:31,762 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: D0EC84A6300E4617A6DFD1D38B5DC7A4
2016-05-17 21:52:31,763 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"], id: 27DB08F568EF4E09B6365C456C48AFD3
2016-05-17 21:52:31,764 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-17 21:52:31,764 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 210 bytes from disk
2016-05-17 21:52:31,764 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:31,764 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:31,765 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2016-05-17 21:52:31,765 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,765 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] D0EC84A6300E4617A6DFD1D38B5DC7A4, for 1 inputs
2016-05-17 21:52:31,766 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:31,767 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E959E740B0D74FE8951993F5CB9EF4D4, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:31,772 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4ae22b2; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@4467d684
2016-05-17 21:52:31,774 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:31,775 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:31,775 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,776 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521949609_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 21:52:31,777 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,777 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E959E740B0D74FE8951993F5CB9EF4D4, mem on close (mb), free: 95, total: 345, max: 672
2016-05-17 21:52:31,777 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BD9CBF0804B84E23A67381209AA8AA79
2016-05-17 21:52:31,778 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:31,779 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:31,779 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:31,779 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:31,780 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:31 Completed running task attempt: attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,781 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:31,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:31,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521951767, Process_Duration=10, Process_End_Time=1463521951777, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 21:52:31,785 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:31,786 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,786 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521949609_0001_00_000003
2016-05-17 21:52:31,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, taskAttemptId=attempt_1463521949609_0001_1_02_000000_0, startTime=1463521951709, finishTime=1463521951783, timeTaken=74, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951767, Process_Duration=10, Process_End_Time=1463521951777, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:31,787 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521949609_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:31,788 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521949609_0001_1_02_000000_0
2016-05-17 21:52:31,788 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:31,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:31,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:TASK_FINISHED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, taskId=task_1463521949609_0001_1_02_000000, startTime=1463521951709, finishTime=1463521951788, timeTaken=79, status=SUCCEEDED, successfulAttemptID=attempt_1463521949609_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951767, Process_Duration=10, Process_End_Time=1463521951777, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:31,790 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521949609_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:31,790 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] : 1
2016-05-17 21:52:31,790 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:31,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:VERTEX_FINISHED]: vertexName=E959E740B0D74FE8951993F5CB9EF4D4, vertexId=vertex_1463521949609_0001_1_02, initRequestedTime=1463521950731, initedTime=1463521950805, startRequestedTime=1463521950883, startedTime=1463521950883, finishTime=1463521951790, timeTaken=907, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521951767, Process_Duration=10, Process_End_Time=1463521951777, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463521951709, firstTasksToStart=[ task_1463521949609_0001_1_02_000000 ], lastTaskFinishTime=1463521951783, lastTasksToFinish=[ task_1463521949609_0001_1_02_000000 ], minTaskDuration=74, maxTaskDuration=74, avgTaskDuration=74.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521949609_0001_1_02_000000 ], longestDurationTasks=[ task_1463521949609_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:31,792 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:31,792 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:31,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521949609_0001_00_000003, stoppedTime=1463521951792, exitStatus=0
2016-05-17 21:52:31,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:31,794 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:31,795 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:31,795 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521949609_0001_1
2016-05-17 21:52:31,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:31,799 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521949609_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 21:52:31,799 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521949609_0001_00_000003: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:31,800 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521949609_0001_00_000003, Exiting
2016-05-17 21:52:31,800 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521949609_0001_00_000003
2016-05-17 21:52:31,800 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521949609_0001_00_000003
2016-05-17 21:52:31,800 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521949609_0001_1
2016-05-17 21:52:31,800 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521949609_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:31,801 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521949609_0001_00_000003 completed successfully
2016-05-17 21:52:31,800 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 27DB08F568EF4E09B6365C456C48AFD3 for vertex: vertex_1463521949609_0001_1_02 [E959E740B0D74FE8951993F5CB9EF4D4]
2016-05-17 21:52:31,812 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521949609_0001_1_01 [BD9CBF0804B84E23A67381209AA8AA79]
2016-05-17 21:52:31,813 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521949609_0001_1_00 [BC12AAA767694F8E8C505DF617C89107]
2016-05-17 21:52:31,820 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521949609_0001_1, queueSize=0
2016-05-17 21:52:31,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:31,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521949609_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521949609_0001_1, startTime=1463521950604, finishTime=1463521951813, timeTaken=1209, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2716, FILE_BYTES_WRITTEN=1138, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=14, REDUCE_INPUT_RECORDS=18, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=36, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1018691584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=26, OUTPUT_BYTES=394, OUTPUT_BYTES_WITH_OVERHEAD=406, OUTPUT_BYTES_PHYSICAL=414, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=414, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=414, SHUFFLE_BYTES_DECOMPRESSED=406, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=414, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=42, MERGE_PHASE_TIME=136, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=31, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390565854755, Process_Duration=32, Process_End_Time=4390565854787, Read_Duration=6, Tuples_Read=28, Tuples_Written=26, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 21:52:31,824 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521949609_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:31,824 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521949609_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:31,827 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521949609_0001_00_000003
2016-05-17 21:52:31,827 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521949609_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:31 Completed Dag: dag_1463521949609_0001_1
2016-05-17 21:52:31,829 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521949609_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:31,829 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521949609_0001_1_post
2016-05-17 21:52:31,829 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:31,831 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:31,837 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463521949609_0001
2016-05-17 21:52:31,841 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:31,841 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:31,841 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:31,841 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:31,841 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:31,843 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:31,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:31,844 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:31,844 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:31,845 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:31,845 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:31,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:31,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:31,851 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/231DEC74A10A4B40BB39FC7A16BD509D/AFA2D5E9F8BA4D689F8D1C0CA92FC6A4/.tez/application_1463521949609_0001
2016-05-17 21:52:31,853 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain/_temporary
2016-05-17 21:52:31,853 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:02.451
2016-05-17 21:52:31,911 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-17 21:52:31,957 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:31,958 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:32,033 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 21:52:32,062 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.055
2016-05-17 21:52:32,063 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:32,063 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:32,064 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:32,089 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 21:52:32,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count
2016-05-17 21:52:32,221 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 21:52:32,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:32,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-17 21:52:32,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-17 21:52:32,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-17 21:52:32,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-17 21:52:32,224 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...splatform/testCount/count
2016-05-17 21:52:32,224 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:32,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 21:52:32,250 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:32,250 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:32,251 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:32,262 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001 doesn't exist and is created
2016-05-17 21:52:32,285 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:32,285 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:32,286 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001
2016-05-17 21:52:32,290 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521952250_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:32,305 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:32,306 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:32,308 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:32,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:32,308 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001/localmode-log-dir
2016-05-17 21:52:32,309 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001/localmode-log-dir/history.txt.appattempt_1463521952250_0001_000000, maxErrors=10
2016-05-17 21:52:32,309 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:32,309 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521952250_0001_000000, appSubmitTime=1463521952290, launchTime=1463521952290
2016-05-17 21:52:32,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:32,319 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59387
2016-05-17 21:52:32,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:32,324 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:32,324 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:32,328 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:32,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521952250_0001_000000, startTime=1463521952331
2016-05-17 21:52:32,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:32,332 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:32,386 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:32,386 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:32,386 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463521952250_0001, dagName=[5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count
2016-05-17 21:52:32,413 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count
2016-05-17 21:52:32,419 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521952250_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001/localmode-log-dir/dag_1463521952250_0001_1.dot
2016-05-17 21:52:32,421 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001/localmode-log-dir/dag_1463521952250_0001_1-tez-dag.pb.txt
2016-05-17 21:52:32,463 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521952250_0001_1
2016-05-17 21:52:32,463 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,464 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count
2016-05-17 21:52:32 Running Dag: dag_1463521952250_0001_1
2016-05-17 21:52:32,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:32,484 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521952250_0001_1, submitTime=1463521952413
2016-05-17 21:52:32,486 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,487 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:32,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:32,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521952250_0001_1, initTime=1463521952485
2016-05-17 21:52:32,488 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521952250_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:32,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_STARTED]: dagID=dag_1463521952250_0001_1, startTime=1463521952489
2016-05-17 21:52:32,490 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:32,490 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521952250_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:32,490 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:32,491 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FFDB880A50474694B112BB25B58188B0 : {E8EAAE3498134BE0AF141663FC08543C={InputName=E8EAAE3498134BE0AF141663FC08543C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:32,491 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E8EAAE3498134BE0AF141663FC08543C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:32,491 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,491 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,491 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,492 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]. Starting root input initializers: 1
2016-05-17 21:52:32,493 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:32,493 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,494 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E8EAAE3498134BE0AF141663FC08543C on vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,496 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463521952250_0001, dagName=[5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count
2016-05-17 21:52:32,503 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:32,504 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,504 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,505 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,505 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3B2A087FB8BC40B4B51AA03410512F76, vertexId=vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:32,506 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3B2A087FB8BC40B4B51AA03410512F76, vertexId=vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,512 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:32,514 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E8EAAE3498134BE0AF141663FC08543C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:32,517 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:32,517 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:32,517 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3B2A087FB8BC40B4B51AA03410512F76, vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:32,518 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3B2A087FB8BC40B4B51AA03410512F76, vertexId=vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,521 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 21:52:32,524 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:32,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, vertexId=vertex_1463521952250_0001_1_01, initRequestedTime=1463521952493, initedTime=1463521952524, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:32,525 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:32,525 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E8EAAE3498134BE0AF141663FC08543C on vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,525 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:32,526 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E8EAAE3498134BE0AF141663FC08543C=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:32,528 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] parallelism set to 1
2016-05-17 21:52:32,528 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:32,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521952250_0001_1_00, eventCount=1
2016-05-17 21:52:32,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:32,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FFDB880A50474694B112BB25B58188B0, vertexId=vertex_1463521952250_0001_1_00, initRequestedTime=1463521952490, initedTime=1463521952530, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:32,532 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] with distanceFromRoot: 0
2016-05-17 21:52:32,532 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463521952250_0001, with dag id: dag_1463521952250_0001_1
2016-05-17 21:52:32,532 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:32,533 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,533 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,535 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:32,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521952250_0001_1_00, startRequestedTime=1463521952533, startedTime=1463521952533
2016-05-17 21:52:32,536 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:32,537 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:32,537 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_STARTED]: vertexName=FFDB880A50474694B112BB25B58188B0, taskId=task_1463521952250_0001_1_00_000000, scheduledTime=1463521952537, launchTime=1463521952537
2016-05-17 21:52:32,538 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:32,538 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521952250_0001_1_00 for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] numStartedSources: 1 numSources: 1
2016-05-17 21:52:32,540 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] with distanceFromRoot: 1
2016-05-17 21:52:32,540 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,553 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FFDB880A50474694B112BB25B58188B0 in vertex: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,554 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EA6F324C2A0443EE8B19EA89C639CEF3 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:32,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521952250_0001_1_01, startRequestedTime=1463521952543, startedTime=1463521952543
2016-05-17 21:52:32,554 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:32,554 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:32,558 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count, VertexName: FFDB880A50474694B112BB25B58188B0, VertexParallelism: 1, TaskAttemptID:attempt_1463521952250_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8EAAE3498134BE0AF141663FC08543C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA6F324C2A0443EE8B19EA89C639CEF3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:32,560 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:32,561 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:32,564 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:32,564 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:32,564 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:32,565 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521952250_0001_00_000001
2016-05-17 21:52:32,565 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:32,571 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521952250_0001_00_000001
2016-05-17 21:52:32,571 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:32,572 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:32,573 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:32,573 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521952250_0001_00_000001, launchTime=1463521952572
2016-05-17 21:52:32,575 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521952250_0001_00_000001
2016-05-17 21:52:32,578 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521952250_0001_1_00_000000_0] to container: [container_1463521952250_0001_00_000001]
2016-05-17 21:52:32,578 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:32,579 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521952250_0001_1_00_000000_0] started. Is using containerId: [container_1463521952250_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:32,579 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:32,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FFDB880A50474694B112BB25B58188B0, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0, startTime=1463521952578, containerId=container_1463521952250_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521952250_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:32,579 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:32,580 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:32,580 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521952250_0001_00_000001 given task: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,581 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521952250_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32 Starting to run new task attempt: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,581 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,581 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,582 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:32,582 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:32,582 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:32,582 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:32,582 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:32,582 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count, VertexName: FFDB880A50474694B112BB25B58188B0, VertexParallelism: 1, TaskAttemptID:attempt_1463521952250_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8EAAE3498134BE0AF141663FC08543C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA6F324C2A0443EE8B19EA89C639CEF3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:32,583 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:32,583 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:32,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E8EAAE3498134BE0AF141663FC08543C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:32,588 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:32,589 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,590 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,590 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EA6F324C2A0443EE8B19EA89C639CEF3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:32,591 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521952250_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:32,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:32,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,597 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:32,601 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:32,601 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,608 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:32,609 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,610 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FFDB880A50474694B112BB25B58188B0, ordinal: 0
2016-05-17 21:52:32,611 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FFDB880A50474694B112BB25B58188B0, mem on start (mb), free: 113, total: 345, max: 672
2016-05-17 21:52:32,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:32,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:32,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:32,611 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:32,612 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:32,612 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:32,612 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:32,613 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:32,613 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:32,613 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E8EAAE3498134BE0AF141663FC08543C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:32,614 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EA6F324C2A0443EE8B19EA89C639CEF3, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:32,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:32,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,615 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:32,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E8EAAE3498134BE0AF141663FC08543C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:32,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:32,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:32,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:32,617 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,623 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:32,625 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@43a08a6b
2016-05-17 21:52:32,625 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:32,627 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: E8EAAE3498134BE0AF141663FC08543C
2016-05-17 21:52:32,627 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 0ABF8795FBD0485EA0C193EDB8452F9B
2016-05-17 21:52:32,627 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 0ABF8795FBD0485EA0C193EDB8452F9B
2016-05-17 21:52:32,630 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@43a08a6b
2016-05-17 21:52:32,632 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:32,632 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:32,632 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2fef37c0; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@23536d6; comparator=cascading.tuple.hadoop.util.TupleComparator@73a31cbb; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:32,637 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:32,638 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:32,638 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:32,760 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:32,761 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:32,761 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:32,761 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:32,762 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:32,762 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:32,763 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:32,763 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FFDB880A50474694B112BB25B58188B0, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:32,766 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,766 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FFDB880A50474694B112BB25B58188B0, mem on close (mb), free: 111, total: 445, max: 672
2016-05-17 21:52:32,766 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:32,767 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:32,767 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 21:52:32,767 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:52:32,772 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:32 Completed running task attempt: attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,777 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:32,778 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521952250_0001_1_00, eventCount=1
2016-05-17 21:52:32,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:32,781 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FFDB880A50474694B112BB25B58188B0, taskAttemptId=attempt_1463521952250_0001_1_00_000000_0, startTime=1463521952578, finishTime=1463521952779, timeTaken=201, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952763, Process_Duration=3, Process_End_Time=1463521952766, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:32,781 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:32,782 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521952250_0001_1_00_000000_0
2016-05-17 21:52:32,782 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:32,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_FINISHED]: vertexName=FFDB880A50474694B112BB25B58188B0, taskId=task_1463521952250_0001_1_00_000000, startTime=1463521952578, finishTime=1463521952782, timeTaken=204, status=SUCCEEDED, successfulAttemptID=attempt_1463521952250_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952763, Process_Duration=3, Process_End_Time=1463521952766, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:32,788 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:32,790 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] : 1
2016-05-17 21:52:32,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:32,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521952763, Process_Duration=3, Process_End_Time=1463521952766, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:52:32,790 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,791 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:32,792 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:32,793 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521952250_0001_00_000001
2016-05-17 21:52:32,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_FINISHED]: vertexName=FFDB880A50474694B112BB25B58188B0, vertexId=vertex_1463521952250_0001_1_00, initRequestedTime=1463521952490, initedTime=1463521952530, startRequestedTime=1463521952533, startedTime=1463521952533, finishTime=1463521952791, timeTaken=258, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952763, Process_Duration=3, Process_End_Time=1463521952766, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463521952578, firstTasksToStart=[ task_1463521952250_0001_1_00_000000 ], lastTaskFinishTime=1463521952779, lastTasksToFinish=[ task_1463521952250_0001_1_00_000000 ], minTaskDuration=201, maxTaskDuration=201, avgTaskDuration=201.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521952250_0001_1_00_000000 ], longestDurationTasks=[ task_1463521952250_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:32,795 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:32,795 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:32,795 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] attempt: attempt_1463521952250_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:32,795 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EA6F324C2A0443EE8B19EA89C639CEF3
2016-05-17 21:52:32,796 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,796 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521952250_0001_00_000001, stoppedTime=1463521952796, exitStatus=0
2016-05-17 21:52:32,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:32,796 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:32,797 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:32,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:32,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_STARTED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, taskId=task_1463521952250_0001_1_01_000000, scheduledTime=1463521952797, launchTime=1463521952797
2016-05-17 21:52:32,798 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:32,798 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521952250_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:32,799 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,799 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:32,799 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count, VertexName: EA6F324C2A0443EE8B19EA89C639CEF3, VertexParallelism: 1, TaskAttemptID:attempt_1463521952250_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FFDB880A50474694B112BB25B58188B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3B2A087FB8BC40B4B51AA03410512F76, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:32,799 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:32,800 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:32,801 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521952250_0001_00_000002
2016-05-17 21:52:32,801 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:32,803 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521952250_0001_00_000002
2016-05-17 21:52:32,803 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:32,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:32,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521952250_0001_00_000002, launchTime=1463521952804
2016-05-17 21:52:32,804 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:32,805 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521952250_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:32,805 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521952250_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:32,805 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521952250_0001_00_000001, Exiting
2016-05-17 21:52:32,805 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521952250_0001_00_000001
2016-05-17 21:52:32,805 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521952250_0001_00_000001
2016-05-17 21:52:32,806 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521952250_0001_00_000002
2016-05-17 21:52:32,807 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521952250_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:32,807 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521952250_0001_00_000001 completed successfully
2016-05-17 21:52:32,807 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521952250_0001_1_01_000000_0] to container: [container_1463521952250_0001_00_000002]
2016-05-17 21:52:32,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:32,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:32,808 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521952250_0001_1_01_000000_0] started. Is using containerId: [container_1463521952250_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:32,809 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:32,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0, startTime=1463521952808, containerId=container_1463521952250_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521952250_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:32,809 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:32,810 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:32,808 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521952250_0001_00_000002 given task: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,810 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521952250_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32 Starting to run new task attempt: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,811 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,811 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,811 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:32,811 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:32,811 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:32,811 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:32,812 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:32,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0] count/(1/1) ...splatform/testCount/count, VertexName: EA6F324C2A0443EE8B19EA89C639CEF3, VertexParallelism: 1, TaskAttemptID:attempt_1463521952250_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FFDB880A50474694B112BB25B58188B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3B2A087FB8BC40B4B51AA03410512F76, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:32,812 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:32,813 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:32,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FFDB880A50474694B112BB25B58188B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:32,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:32,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,823 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521952250_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:32,824 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:32,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3B2A087FB8BC40B4B51AA03410512F76, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:32,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:32,826 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3B2A087FB8BC40B4B51AA03410512F76
2016-05-17 21:52:32,826 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:32,826 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3B2A087FB8BC40B4B51AA03410512F76
2016-05-17 21:52:32,828 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:32,828 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:32,836 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,848 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:32,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3B2A087FB8BC40B4B51AA03410512F76
2016-05-17 21:52:32,849 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EA6F324C2A0443EE8B19EA89C639CEF3, ordinal: 1
2016-05-17 21:52:32,850 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA6F324C2A0443EE8B19EA89C639CEF3, mem on start (mb), free: 103, total: 445, max: 672
2016-05-17 21:52:32,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:32,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:32,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:32,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:32,850 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:32,850 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:32,850 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:32,851 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:32,851 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:32,851 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:32,852 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3B2A087FB8BC40B4B51AA03410512F76, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:32,852 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FFDB880A50474694B112BB25B58188B0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:32,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:32,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FFDB880A50474694B112BB25B58188B0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:32,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:32,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:32,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,854 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:32,855 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:32,855 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FFDB880A50474694B112BB25B58188B0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:32,857 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:32,857 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:32,857 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:32,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:32,859 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,862 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521952250_0001_1_00_000000_0_10012, runDuration: 178816, ]
2016-05-17 21:52:32,864 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521952250_0001_1_00_000000_0_10012] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463521952864, TimeTaken=2, Rate=0.22 MB/s
2016-05-17 21:52:32,864 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,864 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:32,865 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [FFDB880A50474694B112BB25B58188B0] #3 in 3ms
2016-05-17 21:52:32,866 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 21:52:32,866 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FFDB880A50474694B112BB25B58188B0] #3, status:false, isInterrupted:false
2016-05-17 21:52:32,866 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 0ABF8795FBD0485EA0C193EDB8452F9B
2016-05-17 21:52:32,867 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"], id: 3B2A087FB8BC40B4B51AA03410512F76
2016-05-17 21:52:32,867 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-17 21:52:32,867 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 0ABF8795FBD0485EA0C193EDB8452F9B, for 1 inputs
2016-05-17 21:52:32,868 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:32,868 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 21:52:32,869 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:32,869 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:32,869 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 21:52:32,870 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,870 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:32,870 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EA6F324C2A0443EE8B19EA89C639CEF3, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:52:32,872 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7e779b1b; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5e684a7a
2016-05-17 21:52:32,876 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:32,876 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:32,877 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,877 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521952250_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:32,879 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,879 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA6F324C2A0443EE8B19EA89C639CEF3, mem on close (mb), free: 100, total: 445, max: 672
2016-05-17 21:52:32,879 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FFDB880A50474694B112BB25B58188B0
2016-05-17 21:52:32,879 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:32,880 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:32,880 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:32,881 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:32,881 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:32 Completed running task attempt: attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,883 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:32,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:32,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521952871, Process_Duration=7, Process_End_Time=1463521952878, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 21:52:32,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, taskAttemptId=attempt_1463521952250_0001_1_01_000000_0, startTime=1463521952808, finishTime=1463521952884, timeTaken=76, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952871, Process_Duration=7, Process_End_Time=1463521952878, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:32,886 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521952250_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:32,886 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:32,886 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521952250_0001_1_01_000000_0
2016-05-17 21:52:32,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:32,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:TASK_FINISHED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, taskId=task_1463521952250_0001_1_01_000000, startTime=1463521952808, finishTime=1463521952887, timeTaken=79, status=SUCCEEDED, successfulAttemptID=attempt_1463521952250_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952871, Process_Duration=7, Process_End_Time=1463521952878, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:32,889 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521952250_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:32,889 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] : 1
2016-05-17 21:52:32,890 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,890 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:32,890 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,891 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521952250_0001_00_000002
2016-05-17 21:52:32,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:32,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:VERTEX_FINISHED]: vertexName=EA6F324C2A0443EE8B19EA89C639CEF3, vertexId=vertex_1463521952250_0001_1_01, initRequestedTime=1463521952493, initedTime=1463521952524, startRequestedTime=1463521952543, startedTime=1463521952543, finishTime=1463521952890, timeTaken=347, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=467140608, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521952871, Process_Duration=7, Process_End_Time=1463521952878, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463521952808, firstTasksToStart=[ task_1463521952250_0001_1_01_000000 ], lastTaskFinishTime=1463521952884, lastTasksToFinish=[ task_1463521952250_0001_1_01_000000 ], minTaskDuration=76, maxTaskDuration=76, avgTaskDuration=76.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521952250_0001_1_01_000000 ], longestDurationTasks=[ task_1463521952250_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:32,893 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:32,897 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:32,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521952250_0001_00_000002, stoppedTime=1463521952897, exitStatus=0
2016-05-17 21:52:32,898 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:32,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:32,898 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:32,898 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521952250_0001_1
2016-05-17 21:52:32,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521952250_0001_1
2016-05-17 21:52:32,899 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3B2A087FB8BC40B4B51AA03410512F76 for vertex: vertex_1463521952250_0001_1_01 [EA6F324C2A0443EE8B19EA89C639CEF3]
2016-05-17 21:52:32,901 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521952250_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:32,902 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521952250_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:32,902 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521952250_0001_00_000002, Exiting
2016-05-17 21:52:32,902 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521952250_0001_00_000002
2016-05-17 21:52:32,902 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521952250_0001_00_000002
2016-05-17 21:52:32,903 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521952250_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:32,903 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521952250_0001_00_000002 completed successfully
2016-05-17 21:52:32,913 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521952250_0001_1_00 [FFDB880A50474694B112BB25B58188B0]
2016-05-17 21:52:32,916 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521952250_0001_1, queueSize=0
2016-05-17 21:52:32,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:32,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521952250_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521952250_0001_1, startTime=1463521952489, finishTime=1463521952913, timeTaken=424, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=934281216, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043905634, Process_Duration=10, Process_End_Time=2927043905644, Read_Duration=4, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 21:52:32,916 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521952250_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:32,917 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521952250_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:32,917 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521952250_0001_00_000002
2016-05-17 21:52:32,918 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521952250_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:32 Completed Dag: dag_1463521952250_0001_1
2016-05-17 21:52:32,920 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521952250_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:32,921 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521952250_0001_1_post
2016-05-17 21:52:32,921 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:32,921 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:32,926 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463521952250_0001
2016-05-17 21:52:32,929 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:32,929 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:32,929 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:32,929 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:32,929 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:32,930 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:32,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:32,930 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:32,931 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:32,931 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:32,931 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:32,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:32,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:32,936 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5A321C5411F346EB818F35CD6B432274/FDCBB42FA5D743EF9E57D6154EB42DD0/.tez/application_1463521952250_0001
2016-05-17 21:52:32,937 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count/_temporary
2016-05-17 21:52:32,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.717
2016-05-17 21:52:33,022 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount/count
2016-05-17 21:52:33,049 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:33,050 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:33,102 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-17 21:52:33,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-17 21:52:33,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:33,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:33,126 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:33,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:33,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-17 21:52:33,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:33,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:33,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-17 21:52:33,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:33,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:33,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:33,160 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33,160 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:33,177 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:33,182 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:33,182 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:33,183 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:33,196 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001 doesn't exist and is created
2016-05-17 21:52:33,215 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:33,216 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:33,216 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001
2016-05-17 21:52:33,221 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521953182_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:33,233 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:33,234 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:33,236 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:33,236 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:33,236 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001/localmode-log-dir
2016-05-17 21:52:33,236 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001/localmode-log-dir/history.txt.appattempt_1463521953182_0001_000000, maxErrors=10
2016-05-17 21:52:33,237 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:33,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521953182_0001_000000, appSubmitTime=1463521953220, launchTime=1463521953220
2016-05-17 21:52:33,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:33,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:33,268 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51682
2016-05-17 21:52:33,269 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:33,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:33,274 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:33,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521953182_0001_000000, startTime=1463521953278
2016-05-17 21:52:33,278 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:33,278 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:33,316 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:33,316 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:33,316 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463521953182_0001, dagName=[166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33,338 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33,339 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521953182_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001/localmode-log-dir/dag_1463521953182_0001_1.dot
2016-05-17 21:52:33,340 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001/localmode-log-dir/dag_1463521953182_0001_1-tez-dag.pb.txt
2016-05-17 21:52:33,366 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521953182_0001_1
2016-05-17 21:52:33,366 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,366 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33 Running Dag: dag_1463521953182_0001_1
2016-05-17 21:52:33,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:33,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521953182_0001_1, submitTime=1463521953338
2016-05-17 21:52:33,382 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AC05A3849AB84C8E941B50FCD8DFFAD1
2016-05-17 21:52:33,383 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:33,383 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:33,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521953182_0001_1, initTime=1463521953382
2016-05-17 21:52:33,383 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953182_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:33,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_STARTED]: dagID=dag_1463521953182_0001_1, startTime=1463521953385
2016-05-17 21:52:33,385 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:33,385 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953182_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:33,385 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AC05A3849AB84C8E941B50FCD8DFFAD1 : {66D86C6E329645E794BE88DC26DA2EE1={InputName=66D86C6E329645E794BE88DC26DA2EE1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:33,386 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 66D86C6E329645E794BE88DC26DA2EE1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:33,386 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,386 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,386 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,386 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]. Starting root input initializers: 1
2016-05-17 21:52:33,386 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:33,387 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 66D86C6E329645E794BE88DC26DA2EE1 on vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,387 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463521953182_0001, dagName=[166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 21:52:33,387 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:33,389 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:33,391 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 66D86C6E329645E794BE88DC26DA2EE1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:33,392 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:33,392 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:33,395 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:33,395 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 66D86C6E329645E794BE88DC26DA2EE1 on vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,396 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:33,396 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {66D86C6E329645E794BE88DC26DA2EE1=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:33,396 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] parallelism set to 1
2016-05-17 21:52:33,396 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:33,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521953182_0001_1_00, eventCount=1
2016-05-17 21:52:33,397 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,397 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=558FAA5C8ABD4956B9A128A9A68E513F, vertexId=vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:33,397 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=558FAA5C8ABD4956B9A128A9A68E513F, vertexId=vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,399 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=558FAA5C8ABD4956B9A128A9A68E513F, vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:33,400 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=558FAA5C8ABD4956B9A128A9A68E513F, vertexId=vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, vertexId=vertex_1463521953182_0001_1_00, initRequestedTime=1463521953385, initedTime=1463521953402, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:33,402 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] with distanceFromRoot: 0
2016-05-17 21:52:33,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:33,402 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:33,403 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AC05A3849AB84C8E941B50FCD8DFFAD1
2016-05-17 21:52:33,403 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521953182_0001_1_00, startRequestedTime=1463521953403, startedTime=1463521953403
2016-05-17 21:52:33,404 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:33,404 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:33,405 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:33,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:TASK_STARTED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, taskId=task_1463521953182_0001_1_00_000000, scheduledTime=1463521953405, launchTime=1463521953405
2016-05-17 21:52:33,405 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953182_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:33,406 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,406 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap, VertexName: AC05A3849AB84C8E941B50FCD8DFFAD1, VertexParallelism: 1, TaskAttemptID:attempt_1463521953182_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=66D86C6E329645E794BE88DC26DA2EE1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=558FAA5C8ABD4956B9A128A9A68E513F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:33,407 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953182_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:33,407 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:33,408 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:33,409 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:33,409 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:33,409 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521953182_0001_00_000001
2016-05-17 21:52:33,409 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:33,411 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521953182_0001_00_000001
2016-05-17 21:52:33,412 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:33,413 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:33,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:33,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521953182_0001_00_000001, launchTime=1463521953413
2016-05-17 21:52:33,414 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521953182_0001_00_000001
2016-05-17 21:52:33,415 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521953182_0001_1_00_000000_0] to container: [container_1463521953182_0001_00_000001]
2016-05-17 21:52:33,415 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:33,415 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521953182_0001_1_00_000000_0] started. Is using containerId: [container_1463521953182_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:33,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0, startTime=1463521953415, containerId=container_1463521953182_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521953182_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:33,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:33,416 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953182_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:33,416 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953182_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:33,416 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521953182_0001_00_000001 given task: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,416 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521953182_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33 Starting to run new task attempt: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,417 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,417 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,417 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:33,417 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:33,417 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:33,417 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:33,418 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:33,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78] test/(1/1) ...pesplatform/testSwap/swap, VertexName: AC05A3849AB84C8E941B50FCD8DFFAD1, VertexParallelism: 1, TaskAttemptID:attempt_1463521953182_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=66D86C6E329645E794BE88DC26DA2EE1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=558FAA5C8ABD4956B9A128A9A68E513F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:33,418 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:33,420 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521953182_0001, with dag id: dag_1463521953182_0001_1
2016-05-17 21:52:33,421 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:33,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,426 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521953182_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:33,427 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=66D86C6E329645E794BE88DC26DA2EE1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:33,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=558FAA5C8ABD4956B9A128A9A68E513F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:33,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:33,427 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:33,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 558FAA5C8ABD4956B9A128A9A68E513F
2016-05-17 21:52:33,429 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:33,430 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 558FAA5C8ABD4956B9A128A9A68E513F
2016-05-17 21:52:33,431 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,432 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:33,434 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,437 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:33,437 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,444 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AC05A3849AB84C8E941B50FCD8DFFAD1, ordinal: 0
2016-05-17 21:52:33,444 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC05A3849AB84C8E941B50FCD8DFFAD1, mem on start (mb), free: 241, total: 555, max: 672
2016-05-17 21:52:33,444 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:33,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 558FAA5C8ABD4956B9A128A9A68E513F
2016-05-17 21:52:33,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:33,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:33,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:33,446 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:33,446 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:33,446 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:33,447 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:33,447 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:33,447 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:33,447 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:33,448 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 558FAA5C8ABD4956B9A128A9A68E513F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:33,448 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 66D86C6E329645E794BE88DC26DA2EE1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:33,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:33,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 66D86C6E329645E794BE88DC26DA2EE1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:33,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,449 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:33,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:33,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:33,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:33,450 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,450 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:33,451 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@204e1f9c
2016-05-17 21:52:33,452 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@204e1f9c
2016-05-17 21:52:33,453 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:33,453 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:33,453 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:33,454 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 66D86C6E329645E794BE88DC26DA2EE1
2016-05-17 21:52:33,454 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"], id: 558FAA5C8ABD4956B9A128A9A68E513F
2016-05-17 21:52:33,455 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-17 21:52:33,455 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:33,455 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:33,456 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AC05A3849AB84C8E941B50FCD8DFFAD1, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:33,459 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:33,460 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:33,460 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,460 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521953182_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:33,461 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,461 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC05A3849AB84C8E941B50FCD8DFFAD1, mem on close (mb), free: 239, total: 555, max: 672
2016-05-17 21:52:33 Completed running task attempt: attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,462 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:33,463 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:33,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, taskAttemptId=attempt_1463521953182_0001_1_00_000000_0, startTime=1463521953415, finishTime=1463521953463, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:33,464 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953182_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:33,464 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:33,464 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521953182_0001_1_00_000000_0
2016-05-17 21:52:33,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:33,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:TASK_FINISHED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, taskId=task_1463521953182_0001_1_00_000000, startTime=1463521953415, finishTime=1463521953465, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1463521953182_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:33,465 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953182_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:33,466 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] : 1
2016-05-17 21:52:33,466 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-17 21:52:33,467 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:33,467 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:VERTEX_FINISHED]: vertexName=AC05A3849AB84C8E941B50FCD8DFFAD1, vertexId=vertex_1463521953182_0001_1_00, initRequestedTime=1463521953385, initedTime=1463521953402, startRequestedTime=1463521953403, startedTime=1463521953403, finishTime=1463521953466, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463521953415, firstTasksToStart=[ task_1463521953182_0001_1_00_000000 ], lastTaskFinishTime=1463521953463, lastTasksToFinish=[ task_1463521953182_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521953182_0001_1_00_000000 ], longestDurationTasks=[ task_1463521953182_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:33,468 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:33,468 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:33,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521953182_0001_00_000001, stoppedTime=1463521953468, exitStatus=0
2016-05-17 21:52:33,469 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:33,469 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:33,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:33,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:33,468 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521953182_0001_00_000001
2016-05-17 21:52:33,469 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521953182_0001_1
2016-05-17 21:52:33,472 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521953182_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:33,472 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521953182_0001_1
2016-05-17 21:52:33,473 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 558FAA5C8ABD4956B9A128A9A68E513F for vertex: vertex_1463521953182_0001_1_00 [AC05A3849AB84C8E941B50FCD8DFFAD1]
2016-05-17 21:52:33,472 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521953182_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:33,473 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521953182_0001_00_000001, Exiting
2016-05-17 21:52:33,473 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521953182_0001_00_000001
2016-05-17 21:52:33,473 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521953182_0001_00_000001
2016-05-17 21:52:33,476 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521953182_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:33,476 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521953182_0001_00_000001 completed successfully
2016-05-17 21:52:33,484 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521953182_0001_1, queueSize=0
2016-05-17 21:52:33,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:33,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953182_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521953182_0001_1, startTime=1463521953385, finishTime=1463521953483, timeTaken=98, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953456, Process_Duration=5, Process_End_Time=1463521953461, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:33,485 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521953182_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:33,486 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953182_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:33,486 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521953182_0001_00_000001
2016-05-17 21:52:33,486 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953182_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:33 Completed Dag: dag_1463521953182_0001_1
2016-05-17 21:52:33,487 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521953182_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:33,487 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521953182_0001_1_post
2016-05-17 21:52:33,487 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,487 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:33,493 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463521953182_0001
2016-05-17 21:52:33,495 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:33,495 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:33,496 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:33,496 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:33,496 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:33,496 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:33,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:33,497 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:33,497 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:33,498 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:33,498 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:33,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:33,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:33,505 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/166389D848C34CA1BEFEA1349798C680/AA51967701EB4924A8407D303AB2DC78/.tez/application_1463521953182_0001
2016-05-17 21:52:33,506 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap/_temporary
2016-05-17 21:52:33,506 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.347
2016-05-17 21:52:33,545 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-17 21:52:33,586 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:33,586 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:33,630 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 21:52:33,642 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-17 21:52:33,642 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:33,643 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:33,643 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:33,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt
2016-05-17 21:52:33,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-17 21:52:33,679 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:33,679 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 21:52:33,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-17 21:52:33,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:33,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:33,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:33,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33,682 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:33,696 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:33,701 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:33,701 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:33,701 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:33,712 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001 doesn't exist and is created
2016-05-17 21:52:33,734 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:33,734 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:33,735 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001
2016-05-17 21:52:33,739 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521953701_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:33,751 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:33,753 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:33,755 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:33,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:33,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001/localmode-log-dir
2016-05-17 21:52:33,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001/localmode-log-dir/history.txt.appattempt_1463521953701_0001_000000, maxErrors=10
2016-05-17 21:52:33,756 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:33,757 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521953701_0001_000000, appSubmitTime=1463521953738, launchTime=1463521953738
2016-05-17 21:52:33,763 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:33,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:33,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:33,767 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:33,770 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48724
2016-05-17 21:52:33,772 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:33,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521953701_0001_000000, startTime=1463521953774
2016-05-17 21:52:33,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:33,774 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:33,835 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:33,835 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:33,835 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463521953701_0001, dagName=[5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33,856 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33,858 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521953701_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001/localmode-log-dir/dag_1463521953701_0001_1.dot
2016-05-17 21:52:33,859 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001/localmode-log-dir/dag_1463521953701_0001_1-tez-dag.pb.txt
2016-05-17 21:52:33,882 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521953701_0001_1
2016-05-17 21:52:33,882 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,882 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33 Running Dag: dag_1463521953701_0001_1
2016-05-17 21:52:33,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521953701_0001_1, submitTime=1463521953856
2016-05-17 21:52:33,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:33,895 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F9268B2F41C84C10B3211C1CEDE04AA8
2016-05-17 21:52:33,895 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:33,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521953701_0001_1, initTime=1463521953895
2016-05-17 21:52:33,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:33,896 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953701_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:33,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_STARTED]: dagID=dag_1463521953701_0001_1, startTime=1463521953898
2016-05-17 21:52:33,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:33,898 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:33,898 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953701_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:33,899 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F9268B2F41C84C10B3211C1CEDE04AA8 : {DD2DF070A8EF4E368C0654584C76C110={InputName=DD2DF070A8EF4E368C0654584C76C110}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:33,899 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: DD2DF070A8EF4E368C0654584C76C110, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:33,899 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,899 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,899 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,899 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463521953701_0001, dagName=[5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 21:52:33,899 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]. Starting root input initializers: 1
2016-05-17 21:52:33,900 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:33,900 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: DD2DF070A8EF4E368C0654584C76C110 on vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,902 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:33,903 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input DD2DF070A8EF4E368C0654584C76C110 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:33,904 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:33,904 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:33,907 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 412
2016-05-17 21:52:33,907 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: DD2DF070A8EF4E368C0654584C76C110 on vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,907 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:33,908 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {DD2DF070A8EF4E368C0654584C76C110=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:33,908 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] parallelism set to 1
2016-05-17 21:52:33,908 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:33,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521953701_0001_1_00, eventCount=1
2016-05-17 21:52:33,908 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,908 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1EE9BECA5D384CC9827CC8E37EFE0311, vertexId=vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:33,909 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1EE9BECA5D384CC9827CC8E37EFE0311, vertexId=vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,910 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1EE9BECA5D384CC9827CC8E37EFE0311, vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:33,911 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1EE9BECA5D384CC9827CC8E37EFE0311, vertexId=vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,912 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, vertexId=vertex_1463521953701_0001_1_00, initRequestedTime=1463521953899, initedTime=1463521953912, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:33,913 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] with distanceFromRoot: 0
2016-05-17 21:52:33,913 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:33,913 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:33,913 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F9268B2F41C84C10B3211C1CEDE04AA8
2016-05-17 21:52:33,914 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521953701_0001_1_00, startRequestedTime=1463521953913, startedTime=1463521953913
2016-05-17 21:52:33,915 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:33,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:33,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:33,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:TASK_STARTED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, taskId=task_1463521953701_0001_1_00_000000, scheduledTime=1463521953915, launchTime=1463521953915
2016-05-17 21:52:33,916 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953701_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:33,917 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,917 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: F9268B2F41C84C10B3211C1CEDE04AA8, VertexParallelism: 1, TaskAttemptID:attempt_1463521953701_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DD2DF070A8EF4E368C0654584C76C110, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1EE9BECA5D384CC9827CC8E37EFE0311, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:33,918 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953701_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:33,918 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:33,919 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:33,919 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:33,920 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:33,920 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521953701_0001_00_000001
2016-05-17 21:52:33,921 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:33,923 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521953701_0001_00_000001
2016-05-17 21:52:33,923 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:33,924 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521953701_0001, with dag id: dag_1463521953701_0001_1
2016-05-17 21:52:33,924 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:33,925 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:33,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521953701_0001_00_000001, launchTime=1463521953924
2016-05-17 21:52:33,927 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521953701_0001_00_000001
2016-05-17 21:52:33,928 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521953701_0001_1_00_000000_0] to container: [container_1463521953701_0001_00_000001]
2016-05-17 21:52:33,929 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:33,929 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521953701_0001_1_00_000000_0] started. Is using containerId: [container_1463521953701_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:33,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0, startTime=1463521953929, containerId=container_1463521953701_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521953701_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:33,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:33,930 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953701_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:33,930 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953701_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:33,932 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521953701_0001_00_000001 given task: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,932 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521953701_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33 Starting to run new task attempt: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,933 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,933 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,935 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:33,935 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:33,935 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:33,935 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:33,937 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:33,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: F9268B2F41C84C10B3211C1CEDE04AA8, VertexParallelism: 1, TaskAttemptID:attempt_1463521953701_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DD2DF070A8EF4E368C0654584C76C110, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1EE9BECA5D384CC9827CC8E37EFE0311, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:33,937 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:33,939 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,940 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521953701_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:33,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:33,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1EE9BECA5D384CC9827CC8E37EFE0311, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:33,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DD2DF070A8EF4E368C0654584C76C110, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:33,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:33,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,943 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:33,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1EE9BECA5D384CC9827CC8E37EFE0311
2016-05-17 21:52:33,944 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:33,944 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1EE9BECA5D384CC9827CC8E37EFE0311
2016-05-17 21:52:33,946 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:33,946 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,948 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:33,958 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9268B2F41C84C10B3211C1CEDE04AA8, ordinal: 0
2016-05-17 21:52:33,958 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9268B2F41C84C10B3211C1CEDE04AA8, mem on start (mb), free: 163, total: 555, max: 672
2016-05-17 21:52:33,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:33,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:33,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:33,959 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:33,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1EE9BECA5D384CC9827CC8E37EFE0311
2016-05-17 21:52:33,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:33,959 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:33,959 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:33,959 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:33,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:33,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:33,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:33,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:33,960 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DD2DF070A8EF4E368C0654584C76C110, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:33,960 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1EE9BECA5D384CC9827CC8E37EFE0311, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:33,960 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:33,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DD2DF070A8EF4E368C0654584C76C110 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:33,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:33,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:33,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,961 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:33,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:33,962 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,963 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:33,965 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 21:52:33,965 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@19d0954e
2016-05-17 21:52:33,966 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: DD2DF070A8EF4E368C0654584C76C110
2016-05-17 21:52:33,966 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"], id: 1EE9BECA5D384CC9827CC8E37EFE0311
2016-05-17 21:52:33,966 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-17 21:52:33,967 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 21:52:33,967 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:33,967 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@19d0954e
2016-05-17 21:52:33,968 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:33,968 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:33,968 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9268B2F41C84C10B3211C1CEDE04AA8, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:33,970 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:33,970 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:33,971 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,971 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521953701_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:33,972 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,972 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9268B2F41C84C10B3211C1CEDE04AA8, mem on close (mb), free: 161, total: 555, max: 672
2016-05-17 21:52:33 Completed running task attempt: attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,973 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:33,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, taskAttemptId=attempt_1463521953701_0001_1_00_000000_0, startTime=1463521953929, finishTime=1463521953974, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 21:52:33,975 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521953701_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:33,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:33,975 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521953701_0001_1_00_000000_0
2016-05-17 21:52:33,976 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:33,976 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:33,977 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:TASK_FINISHED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, taskId=task_1463521953701_0001_1_00_000000, startTime=1463521953929, finishTime=1463521953975, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463521953701_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 21:52:33,977 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521953701_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:33,977 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] : 1
2016-05-17 21:52:33,978 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,979 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:33,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:VERTEX_FINISHED]: vertexName=F9268B2F41C84C10B3211C1CEDE04AA8, vertexId=vertex_1463521953701_0001_1_00, initRequestedTime=1463521953899, initedTime=1463521953912, startRequestedTime=1463521953913, startedTime=1463521953913, finishTime=1463521953978, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1463521953929, firstTasksToStart=[ task_1463521953701_0001_1_00_000000 ], lastTaskFinishTime=1463521953974, lastTasksToFinish=[ task_1463521953701_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521953701_0001_1_00_000000 ], longestDurationTasks=[ task_1463521953701_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:33,980 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:33,980 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:33,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521953701_0001_00_000001, stoppedTime=1463521953980, exitStatus=0
2016-05-17 21:52:33,981 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:33,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:33,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-05-17 21:52:33,981 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:33,981 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521953701_0001_1
2016-05-17 21:52:33,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521953701_0001_1
2016-05-17 21:52:33,982 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1EE9BECA5D384CC9827CC8E37EFE0311 for vertex: vertex_1463521953701_0001_1_00 [F9268B2F41C84C10B3211C1CEDE04AA8]
2016-05-17 21:52:33,983 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:33,984 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:33,984 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521953701_0001_00_000001
2016-05-17 21:52:33,985 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521953701_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:33,985 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521953701_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:33,985 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521953701_0001_00_000001, Exiting
2016-05-17 21:52:33,985 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521953701_0001_00_000001
2016-05-17 21:52:33,985 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521953701_0001_00_000001
2016-05-17 21:52:33,990 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521953701_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:33,990 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521953701_0001_00_000001 completed successfully
2016-05-17 21:52:33,995 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521953701_0001_1, queueSize=0
2016-05-17 21:52:33,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:33,996 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521953701_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521953701_0001_1, startTime=1463521953898, finishTime=1463521953993, timeTaken=95, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521953968, Process_Duration=4, Process_End_Time=1463521953972, Read_Duration=0, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 21:52:33,996 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521953701_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:33,996 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521953701_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:33,997 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521953701_0001_00_000001
2016-05-17 21:52:33,997 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521953701_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:33 Completed Dag: dag_1463521953701_0001_1
2016-05-17 21:52:33,999 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521953701_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:33,999 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521953701_0001_1_post
2016-05-17 21:52:34,000 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,000 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:34,004 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463521953701_0001
2016-05-17 21:52:34,006 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:34,006 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:34,006 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:34,006 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:34,006 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:34,007 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:34,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:34,007 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:34,007 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:34,008 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:34,008 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:34,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:34,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:34,013 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5DD28D0C70B141A482800B7F3A1909AA/E88AAA9D2ED54E4BA44BBA12E7C01D46/.tez/application_1463521953701_0001
2016-05-17 21:52:34,013 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped/_temporary
2016-05-17 21:52:34,014 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.334
2016-05-17 21:52:34,052 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-17 21:52:34,087 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:34,088 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:34,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 21:52:34,166 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.048
2016-05-17 21:52:34,166 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:34,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:34,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:34,191 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-05-17 21:52:34,208 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 21:52:34,217 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-17 21:52:34,360 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-17 21:52:34,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:34,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:34,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-17 21:52:34,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] parallel execution of steps is enabled: false
2016-05-17 21:52:34,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executing total steps: 1
2016-05-17 21:52:34,363 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] allocating management threads: 1
2016-05-17 21:52:34,364 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34,364 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:34,381 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-17 21:52:34,386 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:34,387 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:34,387 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:34,396 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001 doesn't exist and is created
2016-05-17 21:52:34,415 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:34,416 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:34,416 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001
2016-05-17 21:52:34,421 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521954386_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:34,436 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:34,437 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:34,439 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:34,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:34,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001/localmode-log-dir
2016-05-17 21:52:34,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001/localmode-log-dir/history.txt.appattempt_1463521954386_0001_000000, maxErrors=10
2016-05-17 21:52:34,439 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:34,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521954386_0001_000000, appSubmitTime=1463521954420, launchTime=1463521954420
2016-05-17 21:52:34,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:34,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:34,450 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38515
2016-05-17 21:52:34,451 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:34,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:34,452 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:34,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:34,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521954386_0001_000000, startTime=1463521954454
2016-05-17 21:52:34,455 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:34,516 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:34,516 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:34,517 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463521954386_0001, dagName=[1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34,543 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34,545 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521954386_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001/localmode-log-dir/dag_1463521954386_0001_1.dot
2016-05-17 21:52:34,549 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001/localmode-log-dir/dag_1463521954386_0001_1-tez-dag.pb.txt
2016-05-17 21:52:34,581 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521954386_0001_1
2016-05-17 21:52:34,581 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,582 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34 Running Dag: dag_1463521954386_0001_1
2016-05-17 21:52:34,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521954386_0001_1, submitTime=1463521954543
2016-05-17 21:52:34,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:34,604 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,605 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:34,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521954386_0001_1, initTime=1463521954603
2016-05-17 21:52:34,607 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521954386_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:34,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:34,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_STARTED]: dagID=dag_1463521954386_0001_1, startTime=1463521954608
2016-05-17 21:52:34,608 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:34,609 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521954386_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:34,609 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C470363514604326B704E08A2AE6FAD0 : {9191FB124AAA460AB8A9F7EBBC3E20E9={InputName=9191FB124AAA460AB8A9F7EBBC3E20E9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:34,609 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9191FB124AAA460AB8A9F7EBBC3E20E9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:34,609 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,609 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,610 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,610 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]. Starting root input initializers: 1
2016-05-17 21:52:34,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:34,614 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:34,615 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9191FB124AAA460AB8A9F7EBBC3E20E9 on vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,615 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D00E6B9D4D534639BBBB9445F8224123 : {776565AC6F3E4415AD6BDC4293BCBA86={InputName=776565AC6F3E4415AD6BDC4293BCBA86}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:34,615 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 776565AC6F3E4415AD6BDC4293BCBA86, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:34,616 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,616 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,616 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,616 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]. Starting root input initializers: 1
2016-05-17 21:52:34,617 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463521954386_0001, dagName=[1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 21:52:34,617 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:34,618 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,618 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 776565AC6F3E4415AD6BDC4293BCBA86 on vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,622 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:34,622 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9191FB124AAA460AB8A9F7EBBC3E20E9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:34,623 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:34,623 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:34,626 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,626 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,627 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,627 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CA127B987753482BAD9D4BC705A52257, vertexId=vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:34,627 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CA127B987753482BAD9D4BC705A52257, vertexId=vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,626 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 776565AC6F3E4415AD6BDC4293BCBA86 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:34,629 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:34,630 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:34,626 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:34,633 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:34,633 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CA127B987753482BAD9D4BC705A52257, vertexName=B84D2ABBACF84A739BC24F619B6792E9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:34,634 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CA127B987753482BAD9D4BC705A52257, vertexId=vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,635 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:34,639 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9191FB124AAA460AB8A9F7EBBC3E20E9 on vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,640 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:34,640 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 776565AC6F3E4415AD6BDC4293BCBA86 on vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,642 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:34,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, vertexId=vertex_1463521954386_0001_1_02, initRequestedTime=1463521954618, initedTime=1463521954641, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:34,643 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:34,643 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:34,645 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9191FB124AAA460AB8A9F7EBBC3E20E9=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:34,645 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] parallelism set to 1
2016-05-17 21:52:34,645 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:34,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521954386_0001_1_01, eventCount=1
2016-05-17 21:52:34,649 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:34,649 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C470363514604326B704E08A2AE6FAD0, vertexId=vertex_1463521954386_0001_1_01, initRequestedTime=1463521954609, initedTime=1463521954648, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:34,649 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] with distanceFromRoot: 0
2016-05-17 21:52:34,650 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:34,650 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:34,650 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {776565AC6F3E4415AD6BDC4293BCBA86=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:34,650 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] parallelism set to 1
2016-05-17 21:52:34,651 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:34,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521954386_0001_1_00, eventCount=1
2016-05-17 21:52:34,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:34,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, vertexId=vertex_1463521954386_0001_1_00, initRequestedTime=1463521954615, initedTime=1463521954654, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:34,655 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] with distanceFromRoot: 0
2016-05-17 21:52:34,655 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:34,655 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,655 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,657 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463521954386_0001, with dag id: dag_1463521954386_0001_1
2016-05-17 21:52:34,657 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:34,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521954386_0001_1_01, startRequestedTime=1463521954655, startedTime=1463521954655
2016-05-17 21:52:34,660 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:34,661 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,661 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,661 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:34,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521954386_0001_1_00, startRequestedTime=1463521954661, startedTime=1463521954661
2016-05-17 21:52:34,662 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:34,664 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:34,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_STARTED]: vertexName=C470363514604326B704E08A2AE6FAD0, taskId=task_1463521954386_0001_1_01_000000, scheduledTime=1463521954664, launchTime=1463521954664
2016-05-17 21:52:34,665 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:34,665 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521954386_0001_1_01 for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] numStartedSources: 1 numSources: 2
2016-05-17 21:52:34,665 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] numStartedSources: 1 numSources: 2
2016-05-17 21:52:34,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:34,669 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_STARTED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, taskId=task_1463521954386_0001_1_00_000000, scheduledTime=1463521954669, launchTime=1463521954669
2016-05-17 21:52:34,670 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:34,671 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521954386_0001_1_00 for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] numStartedSources: 2 numSources: 2
2016-05-17 21:52:34,672 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] with distanceFromRoot: 1
2016-05-17 21:52:34,673 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,673 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,673 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D00E6B9D4D534639BBBB9445F8224123 in vertex: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,673 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C470363514604326B704E08A2AE6FAD0 in vertex: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,673 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B84D2ABBACF84A739BC24F619B6792E9 with 2 source tasks and 1 pending tasks
2016-05-17 21:52:34,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:34,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521954386_0001_1_02, startRequestedTime=1463521954673, startedTime=1463521954673
2016-05-17 21:52:34,674 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:34,674 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: C470363514604326B704E08A2AE6FAD0, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9191FB124AAA460AB8A9F7EBBC3E20E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:34,675 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:34,675 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: D00E6B9D4D534639BBBB9445F8224123, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=776565AC6F3E4415AD6BDC4293BCBA86, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:34,675 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:34,675 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:34,676 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:34,676 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:34,677 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:34,677 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:34,677 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521954386_0001_00_000001
2016-05-17 21:52:34,678 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:34,679 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521954386_0001_00_000002
2016-05-17 21:52:34,679 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:34,681 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521954386_0001_00_000001
2016-05-17 21:52:34,681 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:34,682 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:34,682 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:34,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521954386_0001_00_000001, launchTime=1463521954682
2016-05-17 21:52:34,684 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521954386_0001_00_000002
2016-05-17 21:52:34,687 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000001
2016-05-17 21:52:34,687 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:34,689 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521954386_0001_1_01_000000_0] to container: [container_1463521954386_0001_00_000001]
2016-05-17 21:52:34,690 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:34,691 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521954386_0001_1_01_000000_0] started. Is using containerId: [container_1463521954386_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:34,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C470363514604326B704E08A2AE6FAD0, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0, startTime=1463521954690, containerId=container_1463521954386_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521954386_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:34,692 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:34,692 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:34,692 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:34,693 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:34,693 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521954386_0001_00_000002, launchTime=1463521954692
2016-05-17 21:52:34,693 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:34,694 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521954386_0001_00_000001 given task: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,694 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,697 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000002
2016-05-17 21:52:34 Starting to run new task attempt: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,697 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,698 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,698 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:34,699 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:34,699 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:34,699 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:34,699 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:34,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: C470363514604326B704E08A2AE6FAD0, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9191FB124AAA460AB8A9F7EBBC3E20E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:34,699 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:34,704 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:34,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,705 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521954386_0001_1_00_000000_0] to container: [container_1463521954386_0001_00_000002]
2016-05-17 21:52:34,706 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:34,707 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521954386_0001_1_00_000000_0] started. Is using containerId: [container_1463521954386_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:34,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0, startTime=1463521954706, containerId=container_1463521954386_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521954386_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:34,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:34,707 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:34,707 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:34,708 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521954386_0001_00_000002 given task: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,708 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9191FB124AAA460AB8A9F7EBBC3E20E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:34,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:34,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,743 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,745 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521954386_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:34 Starting to run new task attempt: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,748 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,748 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,749 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:34,750 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:34,750 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:34,750 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:34,750 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:34,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: D00E6B9D4D534639BBBB9445F8224123, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=776565AC6F3E4415AD6BDC4293BCBA86, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:34,750 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:34,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:34,751 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:34,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:34,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,753 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:34,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=776565AC6F3E4415AD6BDC4293BCBA86, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:34,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:34,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,754 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:34,756 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,757 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,757 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:34,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B84D2ABBACF84A739BC24F619B6792E9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:34,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:34,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,766 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:34,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,768 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:34,768 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,767 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,770 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:34,768 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521954386_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:34,772 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,773 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,772 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C470363514604326B704E08A2AE6FAD0, ordinal: 0
2016-05-17 21:52:34,774 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C470363514604326B704E08A2AE6FAD0, mem on start (mb), free: 256, total: 554, max: 672
2016-05-17 21:52:34,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:34,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:34,776 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:34,776 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:34,777 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:34,778 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:34,778 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:34,778 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:34,779 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:34,780 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9191FB124AAA460AB8A9F7EBBC3E20E9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:34,784 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B84D2ABBACF84A739BC24F619B6792E9, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:34,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:34,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9191FB124AAA460AB8A9F7EBBC3E20E9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:34,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:34,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:34,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,787 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:34,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:34,776 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D00E6B9D4D534639BBBB9445F8224123, ordinal: 1
2016-05-17 21:52:34,788 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D00E6B9D4D534639BBBB9445F8224123, mem on start (mb), free: 256, total: 554, max: 672
2016-05-17 21:52:34,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:34,788 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:34,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:34,789 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:34,789 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:34,789 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:34,789 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:34,790 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:34,790 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:34,790 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 776565AC6F3E4415AD6BDC4293BCBA86, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:34,790 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B84D2ABBACF84A739BC24F619B6792E9, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:34,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:34,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 776565AC6F3E4415AD6BDC4293BCBA86 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:34,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:34,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:34,788 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:34,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,792 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:34,793 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:34,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,796 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: 9191FB124AAA460AB8A9F7EBBC3E20E9
2016-05-17 21:52:34,796 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:34,796 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:34,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:34,800 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@66d182d7
2016-05-17 21:52:34,801 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,804 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:34,804 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1462f077; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6b991f15; comparator=cascading.tuple.hadoop.util.TupleComparator@48d8fd10; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:34,804 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 776565AC6F3E4415AD6BDC4293BCBA86
2016-05-17 21:52:34,804 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@66d182d7
2016-05-17 21:52:34,806 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:34,806 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:34,806 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:34,807 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:34,807 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:34,805 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:34,808 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:34,818 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2831bb22; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@58f4c9cb; comparator=cascading.tuple.hadoop.util.TupleComparator@2eec5c0d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:34,806 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:34,825 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:34,828 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:34,828 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:34,829 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:34,829 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:34,825 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:34,829 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:34,829 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:34,828 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@63ce2e87
2016-05-17 21:52:34,834 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:34,834 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:34,835 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@63ce2e87
2016-05-17 21:52:34,835 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:34,837 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:34,835 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C470363514604326B704E08A2AE6FAD0, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:34,854 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:34,855 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:34,856 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:34,856 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:34,856 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:34,857 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:34,857 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:34,858 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D00E6B9D4D534639BBBB9445F8224123, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:34,860 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,861 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,862 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D00E6B9D4D534639BBBB9445F8224123, mem on close (mb), free: 53, total: 554, max: 672
2016-05-17 21:52:34,862 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:34,862 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:34,862 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:34,863 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:34,861 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C470363514604326B704E08A2AE6FAD0, mem on close (mb), free: 53, total: 554, max: 672
2016-05-17 21:52:34,865 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:34,865 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:34,865 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:34,867 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:34,869 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:34,872 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:34 Completed running task attempt: attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,875 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:34,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521954386_0001_1_00, eventCount=1
2016-05-17 21:52:34,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, taskAttemptId=attempt_1463521954386_0001_1_00_000000_0, startTime=1463521954706, finishTime=1463521954876, timeTaken=170, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=222, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954858, Process_Duration=3, Process_End_Time=1463521954861, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:34,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:34,877 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:34,877 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:34,877 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521954386_0001_1_00_000000_0
2016-05-17 21:52:34,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:34,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_FINISHED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, taskId=task_1463521954386_0001_1_00_000000, startTime=1463521954706, finishTime=1463521954880, timeTaken=174, status=SUCCEEDED, successfulAttemptID=attempt_1463521954386_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=222, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954858, Process_Duration=3, Process_End_Time=1463521954861, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:34,881 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:34,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=222, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521954858, Process_Duration=3, Process_End_Time=1463521954861, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:34,882 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] : 1
2016-05-17 21:52:34,883 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:34,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:34,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_FINISHED]: vertexName=D00E6B9D4D534639BBBB9445F8224123, vertexId=vertex_1463521954386_0001_1_00, initRequestedTime=1463521954615, initedTime=1463521954654, startRequestedTime=1463521954661, startedTime=1463521954661, finishTime=1463521954883, timeTaken=222, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=222, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954858, Process_Duration=3, Process_End_Time=1463521954861, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521954706, firstTasksToStart=[ task_1463521954386_0001_1_00_000000 ], lastTaskFinishTime=1463521954876, lastTasksToFinish=[ task_1463521954386_0001_1_00_000000 ], minTaskDuration=170, maxTaskDuration=170, avgTaskDuration=170.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521954386_0001_1_00_000000 ], longestDurationTasks=[ task_1463521954386_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:34,885 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:34,886 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:34,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:34,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521954386_0001_00_000002, stoppedTime=1463521954886, exitStatus=0
2016-05-17 21:52:34,887 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:34,887 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] attempt: attempt_1463521954386_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:34,888 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:34,888 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521954386_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:52:34,889 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:34,889 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:34,889 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,890 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000002
2016-05-17 21:52:34,891 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521954386_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:34,891 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:34,891 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521954386_0001_00_000002, Exiting
2016-05-17 21:52:34,891 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521954386_0001_00_000002
2016-05-17 21:52:34,891 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521954386_0001_00_000002
2016-05-17 21:52:34,893 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521954386_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:34,894 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521954386_0001_00_000002 completed successfully
2016-05-17 21:52:34 Completed running task attempt: attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,895 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:34,897 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:34,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521954386_0001_1_01, eventCount=1
2016-05-17 21:52:34,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:34,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C470363514604326B704E08A2AE6FAD0, taskAttemptId=attempt_1463521954386_0001_1_01_000000_0, startTime=1463521954690, finishTime=1463521954899, timeTaken=209, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=31, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954838, Process_Duration=22, Process_End_Time=1463521954860, Read_Duration=20, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:34,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=31, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521954838, Process_Duration=22, Process_End_Time=1463521954860, Read_Duration=20, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:34,900 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:34,900 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,901 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000001
2016-05-17 21:52:34,903 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:34,903 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:34,903 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521954386_0001_1_01_000000_0
2016-05-17 21:52:34,904 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:34,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_FINISHED]: vertexName=C470363514604326B704E08A2AE6FAD0, taskId=task_1463521954386_0001_1_01_000000, startTime=1463521954690, finishTime=1463521954904, timeTaken=214, status=SUCCEEDED, successfulAttemptID=attempt_1463521954386_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=31, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954838, Process_Duration=22, Process_End_Time=1463521954860, Read_Duration=20, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:34,905 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:34,905 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] : 1
2016-05-17 21:52:34,906 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:34,908 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:34,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_FINISHED]: vertexName=C470363514604326B704E08A2AE6FAD0, vertexId=vertex_1463521954386_0001_1_01, initRequestedTime=1463521954609, initedTime=1463521954648, startRequestedTime=1463521954655, startedTime=1463521954655, finishTime=1463521954906, timeTaken=251, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=31, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521954838, Process_Duration=22, Process_End_Time=1463521954860, Read_Duration=20, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521954690, firstTasksToStart=[ task_1463521954386_0001_1_01_000000 ], lastTaskFinishTime=1463521954899, lastTasksToFinish=[ task_1463521954386_0001_1_01_000000 ], minTaskDuration=209, maxTaskDuration=209, avgTaskDuration=209.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521954386_0001_1_01_000000 ], longestDurationTasks=[ task_1463521954386_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:34,910 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:34,911 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:34,912 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:34,912 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521954386_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:34,913 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:34,913 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521954386_0001_00_000001, Exiting
2016-05-17 21:52:34,913 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521954386_0001_00_000001
2016-05-17 21:52:34,913 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521954386_0001_00_000001
2016-05-17 21:52:34,914 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521954386_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:34,914 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521954386_0001_00_000001 completed successfully
2016-05-17 21:52:34,912 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521954386_0001_00_000001, stoppedTime=1463521954911, exitStatus=0
2016-05-17 21:52:34,915 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:34,915 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] attempt: attempt_1463521954386_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:34,916 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B84D2ABBACF84A739BC24F619B6792E9
2016-05-17 21:52:34,916 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,917 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:34,917 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:34,918 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521954386_0001_00_000001
2016-05-17 21:52:34,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_STARTED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, taskId=task_1463521954386_0001_1_02_000000, scheduledTime=1463521954919, launchTime=1463521954919
2016-05-17 21:52:34,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:34,919 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:34,920 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:34,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: B84D2ABBACF84A739BC24F619B6792E9, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D00E6B9D4D534639BBBB9445F8224123, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C470363514604326B704E08A2AE6FAD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CA127B987753482BAD9D4BC705A52257, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:34,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:34,922 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:34,923 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521954386_0001_00_000003
2016-05-17 21:52:34,924 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:34,927 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521954386_0001_00_000003
2016-05-17 21:52:34,928 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:34,931 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:34,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:34,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521954386_0001_00_000003, launchTime=1463521954931
2016-05-17 21:52:34,933 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000003
2016-05-17 21:52:34,934 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521954386_0001_1_02_000000_0] to container: [container_1463521954386_0001_00_000003]
2016-05-17 21:52:34,934 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:34,935 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521954386_0001_1_02_000000_0] started. Is using containerId: [container_1463521954386_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 21:52:34,935 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0, startTime=1463521954935, containerId=container_1463521954386_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521954386_0001_00_000003/teamcity, completedLogs=
2016-05-17 21:52:34,936 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:34,936 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:34,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:34,937 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521954386_0001_00_000003 given task: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34,937 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34 Starting to run new task attempt: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34,939 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34,939 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:34,939 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:34,939 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:34,939 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:34,940 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:34,940 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:34,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: B84D2ABBACF84A739BC24F619B6792E9, VertexParallelism: 1, TaskAttemptID:attempt_1463521954386_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D00E6B9D4D534639BBBB9445F8224123, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C470363514604326B704E08A2AE6FAD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CA127B987753482BAD9D4BC705A52257, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:34,940 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:34,940 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:34,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:34,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D00E6B9D4D534639BBBB9445F8224123, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:34,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:34,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,946 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:34,947 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:34,947 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,949 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521954386_0001_1_02_000000_0 sent events: (0-2)
2016-05-17 21:52:34,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C470363514604326B704E08A2AE6FAD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:34,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:34,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,952 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:34,953 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:34,953 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CA127B987753482BAD9D4BC705A52257, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:34,956 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:34,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:34,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CA127B987753482BAD9D4BC705A52257
2016-05-17 21:52:34,957 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:34,957 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CA127B987753482BAD9D4BC705A52257
2016-05-17 21:52:34,964 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B84D2ABBACF84A739BC24F619B6792E9, ordinal: 2
2016-05-17 21:52:34,964 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B84D2ABBACF84A739BC24F619B6792E9, mem on start (mb), free: 44, total: 554, max: 672
2016-05-17 21:52:34,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:34,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-05-17 21:52:34,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:34,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:34,991 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:34,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CA127B987753482BAD9D4BC705A52257
2016-05-17 21:52:34,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:34,992 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:34,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:34,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:34,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:34,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:34,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-17 21:52:34,993 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-05-17 21:52:34,993 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D00E6B9D4D534639BBBB9445F8224123, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 21:52:34,993 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C470363514604326B704E08A2AE6FAD0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 21:52:34,993 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CA127B987753482BAD9D4BC705A52257, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:34,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:34,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D00E6B9D4D534639BBBB9445F8224123 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:34,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C470363514604326B704E08A2AE6FAD0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:34,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-05-17 21:52:34,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-05-17 21:52:34,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,994 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:34,994 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:34,995 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:34,995 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D00E6B9D4D534639BBBB9445F8224123 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:34,995 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:34,995 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 21:52:34,995 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:34,995 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C470363514604326B704E08A2AE6FAD0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:34,995 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:34,995 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 21:52:34,996 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:34,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:34,999 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:34,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:34,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:35,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:35,000 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,004 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521954386_0001_1_00_000000_0_10027, runDuration: 107533, ]
2016-05-17 21:52:35,005 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521954386_0001_1_01_000000_0_10024, runDuration: 137200, ]
2016-05-17 21:52:35,006 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]]
2016-05-17 21:52:35,007 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] streamed: true, id: 866FFC915F87476284463FCA2CD8A30D
2016-05-17 21:52:35,007 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"], id: CA127B987753482BAD9D4BC705A52257
2016-05-17 21:52:35,008 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-17 21:52:35,008 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 866FFC915F87476284463FCA2CD8A30D, for 2 inputs
2016-05-17 21:52:35,009 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 866FFC915F87476284463FCA2CD8A30D, for 2 inputs
2016-05-17 21:52:35,010 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521954386_0001_1_01_000000_0_10024] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521955010, TimeTaken=3, Rate=0.03 MB/s
2016-05-17 21:52:35,010 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:35,011 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:35,011 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C470363514604326B704E08A2AE6FAD0] #4 in 4ms
2016-05-17 21:52:35,017 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C470363514604326B704E08A2AE6FAD0] #4, status:false, isInterrupted:false
2016-05-17 21:52:35,018 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521954386_0001_1_00_000000_0_10027] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521955018, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 21:52:35,018 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:35,019 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:35,019 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D00E6B9D4D534639BBBB9445F8224123] #5 in 2ms
2016-05-17 21:52:35,019 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D00E6B9D4D534639BBBB9445F8224123] #5, status:false, isInterrupted:false
2016-05-17 21:52:35,021 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:35,023 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:35,023 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,023 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:35,023 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:35,023 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:35,024 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:35,025 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:35,025 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,025 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:35,026 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:35,026 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:35,030 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 21:52:35,030 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 21:52:35,032 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:35,033 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:35,036 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B84D2ABBACF84A739BC24F619B6792E9, all 2 inputs ready in: 00:00:00.000
2016-05-17 21:52:35,040 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5813c035; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@78b004bf
2016-05-17 21:52:35,043 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@6c07d57f; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7651bc41
2016-05-17 21:52:35,048 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:35,049 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:35,050 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,051 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521954386_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 21:52:35,052 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,052 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B84D2ABBACF84A739BC24F619B6792E9, mem on close (mb), free: 259, total: 555, max: 672
2016-05-17 21:52:35,053 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D00E6B9D4D534639BBBB9445F8224123
2016-05-17 21:52:35,053 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:35,054 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:35,055 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,055 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:35,055 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:35,056 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C470363514604326B704E08A2AE6FAD0
2016-05-17 21:52:35,056 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:35,058 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:35,058 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,058 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:35,059 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:35 Completed running task attempt: attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,060 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:35,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, taskAttemptId=attempt_1463521954386_0001_1_02_000000_0, startTime=1463521954935, finishTime=1463521955062, timeTaken=127, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955037, Process_Duration=15, Process_End_Time=1463521955052, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:35,062 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521954386_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:35,063 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:35,063 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521954386_0001_1_02_000000_0
2016-05-17 21:52:35,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:TASK_FINISHED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, taskId=task_1463521954386_0001_1_02_000000, startTime=1463521954935, finishTime=1463521955063, timeTaken=128, status=SUCCEEDED, successfulAttemptID=attempt_1463521954386_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955037, Process_Duration=15, Process_End_Time=1463521955052, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:35,064 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521954386_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:35,064 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] : 1
2016-05-17 21:52:35,064 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:35,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:35,067 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:35,069 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:VERTEX_FINISHED]: vertexName=B84D2ABBACF84A739BC24F619B6792E9, vertexId=vertex_1463521954386_0001_1_02, initRequestedTime=1463521954618, initedTime=1463521954641, startRequestedTime=1463521954673, startedTime=1463521954673, finishTime=1463521955064, timeTaken=391, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955037, Process_Duration=15, Process_End_Time=1463521955052, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463521954935, firstTasksToStart=[ task_1463521954386_0001_1_02_000000 ], lastTaskFinishTime=1463521955062, lastTasksToFinish=[ task_1463521954386_0001_1_02_000000 ], minTaskDuration=127, maxTaskDuration=127, avgTaskDuration=127.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521954386_0001_1_02_000000 ], longestDurationTasks=[ task_1463521954386_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:35,070 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:35,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:35,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521955037, Process_Duration=15, Process_End_Time=1463521955052, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 21:52:35,073 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:35,073 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,075 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521954386_0001_00_000003
2016-05-17 21:52:35,078 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:35,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521954386_0001_00_000003, stoppedTime=1463521955078, exitStatus=0
2016-05-17 21:52:35,078 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:35,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:35,081 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:35,081 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521954386_0001_1
2016-05-17 21:52:35,081 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521954386_0001_1
2016-05-17 21:52:35,081 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CA127B987753482BAD9D4BC705A52257 for vertex: vertex_1463521954386_0001_1_02 [B84D2ABBACF84A739BC24F619B6792E9]
2016-05-17 21:52:35,088 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521954386_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 21:52:35,088 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521954386_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:35,089 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521954386_0001_00_000003, Exiting
2016-05-17 21:52:35,089 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521954386_0001_00_000003
2016-05-17 21:52:35,089 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521954386_0001_00_000003
2016-05-17 21:52:35,090 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521954386_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:35,090 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521954386_0001_00_000003 completed successfully
2016-05-17 21:52:35,093 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521954386_0001_1_01 [C470363514604326B704E08A2AE6FAD0]
2016-05-17 21:52:35,093 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521954386_0001_1_00 [D00E6B9D4D534639BBBB9445F8224123]
2016-05-17 21:52:35,096 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521954386_0001_1, queueSize=0
2016-05-17 21:52:35,096 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:35,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521954386_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521954386_0001_1, startTime=1463521954608, finishTime=1463521955093, timeTaken=485, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=470, FILE_BYTES_WRITTEN=8276, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=51, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1743781888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390565864733, Process_Duration=40, Process_End_Time=4390565864773, Read_Duration=23, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-17 21:52:35,097 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521954386_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:35,097 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521954386_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:35,100 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521954386_0001_00_000003
2016-05-17 21:52:35,100 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521954386_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:35 Completed Dag: dag_1463521954386_0001_1
2016-05-17 21:52:35,101 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521954386_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:35,102 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521954386_0001_1_post
2016-05-17 21:52:35,102 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,102 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:35,107 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463521954386_0001
2016-05-17 21:52:35,110 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:35,110 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:35,110 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:35,110 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:35,110 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:35,113 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:35,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:35,116 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:35,116 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:35,116 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:35,117 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:35,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:35,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:35,126 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1740B37E7551475DB433EC4D9C445E6A/DC3C3FE5F4F34F0F871E875F8E91FA8D/.tez/application_1463521954386_0001
2016-05-17 21:52:35,127 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown/_temporary
2016-05-17 21:52:35,127 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] completed in: 00:00.766
2016-05-17 21:52:35,211 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-17 21:52:35,237 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:35,237 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:35,290 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 21:52:35,303 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-05-17 21:52:35,304 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:35,304 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:35,305 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:35,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:35,335 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-17 21:52:35,386 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:35,387 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:35,387 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-17 21:52:35,387 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:35,387 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:35,388 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:35,388 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...leRelative/simplerelative
2016-05-17 21:52:35,389 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:35,401 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:35,407 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:35,408 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:35,408 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:35,417 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001 doesn't exist and is created
2016-05-17 21:52:35,434 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:35,435 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:35,435 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001
2016-05-17 21:52:35,438 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521955407_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:35,450 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:35,451 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:35,452 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:35,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:35,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001/localmode-log-dir
2016-05-17 21:52:35,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001/localmode-log-dir/history.txt.appattempt_1463521955407_0001_000000, maxErrors=10
2016-05-17 21:52:35,453 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:35,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521955407_0001_000000, appSubmitTime=1463521955438, launchTime=1463521955438
2016-05-17 21:52:35,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:35,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:35,464 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:35,464 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48986
2016-05-17 21:52:35,466 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:35,467 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:35,469 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521955407_0001_000000, startTime=1463521955469
2016-05-17 21:52:35,469 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:35,469 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:35,535 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:35,535 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:35,535 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463521955407_0001, dagName=[3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative
2016-05-17 21:52:35,556 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative
2016-05-17 21:52:35,558 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521955407_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001/localmode-log-dir/dag_1463521955407_0001_1.dot
2016-05-17 21:52:35,559 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001/localmode-log-dir/dag_1463521955407_0001_1-tez-dag.pb.txt
2016-05-17 21:52:35,571 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521955407_0001_1
2016-05-17 21:52:35,571 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,572 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative
2016-05-17 21:52:35 Running Dag: dag_1463521955407_0001_1
2016-05-17 21:52:35,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521955407_0001_1, submitTime=1463521955556
2016-05-17 21:52:35,585 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:35,586 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,586 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:35,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:35,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521955407_0001_1, initTime=1463521955585
2016-05-17 21:52:35,587 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521955407_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:35,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_STARTED]: dagID=dag_1463521955407_0001_1, startTime=1463521955588
2016-05-17 21:52:35,588 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:35,588 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:35,588 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521955407_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:35,589 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E1F5DCBD4B27474A9A5DFE6A8726B3EB : {B2EAB128DFDC4181BFA9D885CEB348C3={InputName=B2EAB128DFDC4181BFA9D885CEB348C3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:35,589 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B2EAB128DFDC4181BFA9D885CEB348C3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:35,589 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,589 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,590 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,590 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]. Starting root input initializers: 1
2016-05-17 21:52:35,590 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463521955407_0001, dagName=[3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative
2016-05-17 21:52:35,590 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:35,591 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,591 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B2EAB128DFDC4181BFA9D885CEB348C3 on vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,593 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:35,593 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B2EAB128DFDC4181BFA9D885CEB348C3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:35,594 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:35,594 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:35,595 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:35,595 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,595 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,595 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,595 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A1403853B0D3429EAC6D6BD22BC4D2D8, vertexId=vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:35,596 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A1403853B0D3429EAC6D6BD22BC4D2D8, vertexId=vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,597 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:35,598 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B2EAB128DFDC4181BFA9D885CEB348C3 on vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,600 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A1403853B0D3429EAC6D6BD22BC4D2D8, vertexName=F6AB05418EB14BBBA8BA4F876059A456, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:35,601 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A1403853B0D3429EAC6D6BD22BC4D2D8, vertexId=vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:35,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, vertexId=vertex_1463521955407_0001_1_01, initRequestedTime=1463521955591, initedTime=1463521955603, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:35,603 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:35,603 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:35,604 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B2EAB128DFDC4181BFA9D885CEB348C3=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:35,604 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] parallelism set to 1
2016-05-17 21:52:35,604 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:35,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521955407_0001_1_00, eventCount=1
2016-05-17 21:52:35,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:35,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, vertexId=vertex_1463521955407_0001_1_00, initRequestedTime=1463521955589, initedTime=1463521955605, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:35,606 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] with distanceFromRoot: 0
2016-05-17 21:52:35,607 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:35,607 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,607 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521955407_0001_1_00, startRequestedTime=1463521955607, startedTime=1463521955607
2016-05-17 21:52:35,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:35,608 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:35,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:35,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_STARTED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, taskId=task_1463521955407_0001_1_00_000000, scheduledTime=1463521955608, launchTime=1463521955608
2016-05-17 21:52:35,609 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:35,609 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521955407_0001_1_00 for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] numStartedSources: 1 numSources: 1
2016-05-17 21:52:35,609 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] with distanceFromRoot: 1
2016-05-17 21:52:35,609 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,609 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E1F5DCBD4B27474A9A5DFE6A8726B3EB in vertex: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,610 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F6AB05418EB14BBBA8BA4F876059A456 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:35,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521955407_0001_1_01, startRequestedTime=1463521955609, startedTime=1463521955609
2016-05-17 21:52:35,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:35,611 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:35,611 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative, VertexName: E1F5DCBD4B27474A9A5DFE6A8726B3EB, VertexParallelism: 1, TaskAttemptID:attempt_1463521955407_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B2EAB128DFDC4181BFA9D885CEB348C3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F6AB05418EB14BBBA8BA4F876059A456, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:35,612 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:35,612 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:35,613 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:35,613 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521955407_0001, with dag id: dag_1463521955407_0001_1
2016-05-17 21:52:35,613 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:35,613 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:35,614 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521955407_0001_00_000001
2016-05-17 21:52:35,614 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:35,616 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521955407_0001_00_000001
2016-05-17 21:52:35,616 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:35,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:35,617 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:35,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521955407_0001_00_000001, launchTime=1463521955617
2016-05-17 21:52:35,620 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521955407_0001_00_000001
2016-05-17 21:52:35,622 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521955407_0001_1_00_000000_0] to container: [container_1463521955407_0001_00_000001]
2016-05-17 21:52:35,622 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:35,623 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521955407_0001_1_00_000000_0] started. Is using containerId: [container_1463521955407_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:35,624 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:35,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0, startTime=1463521955623, containerId=container_1463521955407_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521955407_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:35,624 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:35,624 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:35,627 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521955407_0001_00_000001 given task: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,627 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521955407_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35 Starting to run new task attempt: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,629 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,630 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,630 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:35,631 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:35,631 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:35,632 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:35,632 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:35,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative, VertexName: E1F5DCBD4B27474A9A5DFE6A8726B3EB, VertexParallelism: 1, TaskAttemptID:attempt_1463521955407_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B2EAB128DFDC4181BFA9D885CEB348C3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F6AB05418EB14BBBA8BA4F876059A456, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:35,633 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:35,634 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:35,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B2EAB128DFDC4181BFA9D885CEB348C3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:35,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:35,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,641 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,641 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521955407_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:35,644 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:35,644 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F6AB05418EB14BBBA8BA4F876059A456, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:35,644 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:35,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:35,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,647 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:35,647 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,654 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E1F5DCBD4B27474A9A5DFE6A8726B3EB, ordinal: 0
2016-05-17 21:52:35,654 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E1F5DCBD4B27474A9A5DFE6A8726B3EB, mem on start (mb), free: 163, total: 555, max: 672
2016-05-17 21:52:35,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:35,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:35,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:35,655 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:35,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:35,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:35,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:35,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:35,656 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:35,656 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B2EAB128DFDC4181BFA9D885CEB348C3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:35,656 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F6AB05418EB14BBBA8BA4F876059A456, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:35,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:35,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B2EAB128DFDC4181BFA9D885CEB348C3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:35,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:35,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:35,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,657 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:35,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:35,658 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,660 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:35,661 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@482ac91d
2016-05-17 21:52:35,662 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:35,663 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: B2EAB128DFDC4181BFA9D885CEB348C3
2016-05-17 21:52:35,663 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 22DB2FDF46894B7098BD96D37589992D
2016-05-17 21:52:35,663 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@482ac91d
2016-05-17 21:52:35,664 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:35,664 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:35,664 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 22DB2FDF46894B7098BD96D37589992D
2016-05-17 21:52:35,667 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6dfadcb; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@470aedf3; comparator=cascading.tuple.hadoop.util.TupleComparator@1e8c10f5; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:35,667 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:35,668 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:35,668 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:35,682 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:35,683 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:35,683 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:35,683 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:35,683 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:35,684 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:35,684 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:35,685 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E1F5DCBD4B27474A9A5DFE6A8726B3EB, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:35,687 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,688 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E1F5DCBD4B27474A9A5DFE6A8726B3EB, mem on close (mb), free: 61, total: 555, max: 672
2016-05-17 21:52:35,688 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:35,688 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:35,688 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 21:52:35,689 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 21:52:35,695 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:35 Completed running task attempt: attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,700 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:35,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521955407_0001_1_00, eventCount=1
2016-05-17 21:52:35,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521955685, Process_Duration=2, Process_End_Time=1463521955687, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 21:52:35,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:35,702 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, taskAttemptId=attempt_1463521955407_0001_1_00_000000_0, startTime=1463521955623, finishTime=1463521955701, timeTaken=78, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955685, Process_Duration=2, Process_End_Time=1463521955687, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:35,702 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:35,702 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:35,702 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521955407_0001_1_00_000000_0
2016-05-17 21:52:35,703 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:35,704 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:35,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_FINISHED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, taskId=task_1463521955407_0001_1_00_000000, startTime=1463521955623, finishTime=1463521955703, timeTaken=80, status=SUCCEEDED, successfulAttemptID=attempt_1463521955407_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955685, Process_Duration=2, Process_End_Time=1463521955687, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:35,704 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:35,705 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] : 1
2016-05-17 21:52:35,705 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,706 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:35,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_FINISHED]: vertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, vertexId=vertex_1463521955407_0001_1_00, initRequestedTime=1463521955589, initedTime=1463521955605, startRequestedTime=1463521955607, startedTime=1463521955607, finishTime=1463521955705, timeTaken=98, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955685, Process_Duration=2, Process_End_Time=1463521955687, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463521955623, firstTasksToStart=[ task_1463521955407_0001_1_00_000000 ], lastTaskFinishTime=1463521955701, lastTasksToFinish=[ task_1463521955407_0001_1_00_000000 ], minTaskDuration=78, maxTaskDuration=78, avgTaskDuration=78.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521955407_0001_1_00_000000 ], longestDurationTasks=[ task_1463521955407_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:35,707 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:35,707 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:35,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521955407_0001_00_000001, stoppedTime=1463521955707, exitStatus=0
2016-05-17 21:52:35,708 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521955407_0001_00_000001
2016-05-17 21:52:35,708 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:35,708 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] attempt: attempt_1463521955407_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:35,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:35,709 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F6AB05418EB14BBBA8BA4F876059A456
2016-05-17 21:52:35,709 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,708 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521955407_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:35,710 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521955407_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:35,710 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:35,710 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521955407_0001_00_000001, Exiting
2016-05-17 21:52:35,710 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521955407_0001_00_000001
2016-05-17 21:52:35,710 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521955407_0001_00_000001
2016-05-17 21:52:35,711 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521955407_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:35,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_STARTED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, taskId=task_1463521955407_0001_1_01_000000, scheduledTime=1463521955711, launchTime=1463521955711
2016-05-17 21:52:35,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:35,712 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:35,713 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,713 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521955407_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:35,713 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:35,713 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521955407_0001_00_000001 completed successfully
2016-05-17 21:52:35,713 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative, VertexName: F6AB05418EB14BBBA8BA4F876059A456, VertexParallelism: 1, TaskAttemptID:attempt_1463521955407_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A1403853B0D3429EAC6D6BD22BC4D2D8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:35,713 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:35,714 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:35,714 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:35,715 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521955407_0001_00_000002
2016-05-17 21:52:35,715 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:35,717 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521955407_0001_00_000002
2016-05-17 21:52:35,718 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:35,720 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:35,720 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:35,721 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521955407_0001_00_000002, launchTime=1463521955720
2016-05-17 21:52:35,721 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521955407_0001_00_000002
2016-05-17 21:52:35,722 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521955407_0001_1_01_000000_0] to container: [container_1463521955407_0001_00_000002]
2016-05-17 21:52:35,722 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:35,724 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521955407_0001_00_000002 given task: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,725 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521955407_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,726 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521955407_0001_1_01_000000_0] started. Is using containerId: [container_1463521955407_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:35,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0, startTime=1463521955726, containerId=container_1463521955407_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521955407_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:35,726 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:35,726 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:35,726 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:35 Starting to run new task attempt: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,727 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,727 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,728 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:35,728 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:35,728 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:35,728 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:35,729 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:35,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97] test/(1/1) ...leRelative/simplerelative, VertexName: F6AB05418EB14BBBA8BA4F876059A456, VertexParallelism: 1, TaskAttemptID:attempt_1463521955407_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A1403853B0D3429EAC6D6BD22BC4D2D8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:35,730 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:35,730 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:35,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,732 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521955407_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:35,734 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A1403853B0D3429EAC6D6BD22BC4D2D8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:35,734 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:35,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E1F5DCBD4B27474A9A5DFE6A8726B3EB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:35,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:35,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,734 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A1403853B0D3429EAC6D6BD22BC4D2D8
2016-05-17 21:52:35,737 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:35,737 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:35,737 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:35,738 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,737 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A1403853B0D3429EAC6D6BD22BC4D2D8
2016-05-17 21:52:35,738 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:35,749 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F6AB05418EB14BBBA8BA4F876059A456, ordinal: 1
2016-05-17 21:52:35,749 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6AB05418EB14BBBA8BA4F876059A456, mem on start (mb), free: 55, total: 555, max: 672
2016-05-17 21:52:35,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:35,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:35,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:35,752 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:35,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A1403853B0D3429EAC6D6BD22BC4D2D8
2016-05-17 21:52:35,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:35,753 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:35,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:35,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:35,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:35,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:35,754 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:35,754 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E1F5DCBD4B27474A9A5DFE6A8726B3EB, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:35,754 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A1403853B0D3429EAC6D6BD22BC4D2D8, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:35,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:35,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E1F5DCBD4B27474A9A5DFE6A8726B3EB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:35,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:35,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:35,755 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:35,756 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:35,756 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E1F5DCBD4B27474A9A5DFE6A8726B3EB with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:35,756 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:35,757 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:35,757 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:35,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:35,758 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,760 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521955407_0001_1_00_000000_0_10034, runDuration: 52342, ]
2016-05-17 21:52:35,762 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:35,764 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 22DB2FDF46894B7098BD96D37589992D
2016-05-17 21:52:35,764 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"], id: A1403853B0D3429EAC6D6BD22BC4D2D8
2016-05-17 21:52:35,765 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-17 21:52:35,765 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 22DB2FDF46894B7098BD96D37589992D, for 1 inputs
2016-05-17 21:52:35,766 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521955407_0001_1_00_000000_0_10034] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463521955766, TimeTaken=1, Rate=0.19 MB/s
2016-05-17 21:52:35,766 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,767 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:35,767 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E1F5DCBD4B27474A9A5DFE6A8726B3EB] #6 in 2ms
2016-05-17 21:52:35,768 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E1F5DCBD4B27474A9A5DFE6A8726B3EB] #6, status:false, isInterrupted:false
2016-05-17 21:52:35,768 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:35,769 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 21:52:35,769 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,769 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:35,769 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 21:52:35,770 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,770 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:35,770 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F6AB05418EB14BBBA8BA4F876059A456, all 1 inputs ready in: 00:00:00.005
2016-05-17 21:52:35,772 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@63a7de79; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@21ff0d0c
2016-05-17 21:52:35,775 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:35,775 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:35,776 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,776 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521955407_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:35,777 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,777 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6AB05418EB14BBBA8BA4F876059A456, mem on close (mb), free: 52, total: 555, max: 672
2016-05-17 21:52:35,777 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E1F5DCBD4B27474A9A5DFE6A8726B3EB
2016-05-17 21:52:35,778 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:35,778 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:35,779 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:35,779 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:35,779 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:35 Completed running task attempt: attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,780 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:35,782 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:35,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521955771, Process_Duration=6, Process_End_Time=1463521955777, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 21:52:35,783 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:35,784 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,784 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521955407_0001_00_000002
2016-05-17 21:52:35,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, taskAttemptId=attempt_1463521955407_0001_1_01_000000_0, startTime=1463521955726, finishTime=1463521955782, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955771, Process_Duration=6, Process_End_Time=1463521955777, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:35,784 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521955407_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:35,784 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521955407_0001_1_01_000000_0
2016-05-17 21:52:35,784 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:35,785 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:35,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:TASK_FINISHED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, taskId=task_1463521955407_0001_1_01_000000, startTime=1463521955726, finishTime=1463521955784, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1463521955407_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955771, Process_Duration=6, Process_End_Time=1463521955777, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:35,785 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521955407_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:35,785 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] : 1
2016-05-17 21:52:35,785 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:35,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:VERTEX_FINISHED]: vertexName=F6AB05418EB14BBBA8BA4F876059A456, vertexId=vertex_1463521955407_0001_1_01, initRequestedTime=1463521955591, initedTime=1463521955603, startRequestedTime=1463521955609, startedTime=1463521955609, finishTime=1463521955786, timeTaken=177, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521955771, Process_Duration=6, Process_End_Time=1463521955777, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463521955726, firstTasksToStart=[ task_1463521955407_0001_1_01_000000 ], lastTaskFinishTime=1463521955782, lastTasksToFinish=[ task_1463521955407_0001_1_01_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521955407_0001_1_01_000000 ], longestDurationTasks=[ task_1463521955407_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:35,789 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:35,789 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:35,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521955407_0001_00_000002, stoppedTime=1463521955789, exitStatus=0
2016-05-17 21:52:35,789 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:35,790 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:35,790 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521955407_0001_1
2016-05-17 21:52:35,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521955407_0001_1
2016-05-17 21:52:35,790 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A1403853B0D3429EAC6D6BD22BC4D2D8 for vertex: vertex_1463521955407_0001_1_01 [F6AB05418EB14BBBA8BA4F876059A456]
2016-05-17 21:52:35,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:35,795 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521955407_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:35,795 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521955407_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:35,795 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521955407_0001_00_000002, Exiting
2016-05-17 21:52:35,795 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521955407_0001_00_000002
2016-05-17 21:52:35,795 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521955407_0001_00_000002
2016-05-17 21:52:35,797 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521955407_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:35,797 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521955407_0001_00_000002 completed successfully
2016-05-17 21:52:35,799 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521955407_0001_1_00 [E1F5DCBD4B27474A9A5DFE6A8726B3EB]
2016-05-17 21:52:35,801 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521955407_0001_1, queueSize=0
2016-05-17 21:52:35,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:35,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521955407_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521955407_0001_1, startTime=1463521955588, finishTime=1463521955799, timeTaken=211, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1163919360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043911456, Process_Duration=8, Process_End_Time=2927043911464, Read_Duration=3, Tuples_Read=20, Tuples_Written=18, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 21:52:35,803 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521955407_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:35,803 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521955407_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:35,804 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521955407_0001_00_000002
2016-05-17 21:52:35,804 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521955407_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:35 Completed Dag: dag_1463521955407_0001_1
2016-05-17 21:52:35,805 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521955407_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:35,805 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521955407_0001_1_post
2016-05-17 21:52:35,805 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:35,806 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:35,811 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463521955407_0001
2016-05-17 21:52:35,814 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:35,814 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:35,814 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:35,814 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:35,814 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:35,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:35,815 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:35,816 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:35,816 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:35,817 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:35,817 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:35,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:35,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:35,822 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3EAB39C8E0BB4E22BD262C1282F28A09/F523A22EC0B34D69A0E2E5A0E4DA5A97/.tez/application_1463521955407_0001
2016-05-17 21:52:35,823 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative/_temporary
2016-05-17 21:52:35,823 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.437
2016-05-17 21:52:35,879 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-17 21:52:36,006 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:36,007 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:36,062 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-17 21:52:36,073 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-05-17 21:52:36,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:36,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:36,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:36,091 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 21:52:36,107 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-17 21:52:36,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:36,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:36,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-17 21:52:36,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:36,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:36,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:36,172 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36,172 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:36,191 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:36,196 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:36,197 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:36,197 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:36,206 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001 doesn't exist and is created
2016-05-17 21:52:36,223 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:36,223 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:36,223 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001
2016-05-17 21:52:36,227 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521956197_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:36,238 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:36,239 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:36,241 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:36,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:36,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001/localmode-log-dir
2016-05-17 21:52:36,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001/localmode-log-dir/history.txt.appattempt_1463521956197_0001_000000, maxErrors=10
2016-05-17 21:52:36,242 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:36,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521956197_0001_000000, appSubmitTime=1463521956227, launchTime=1463521956227
2016-05-17 21:52:36,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:36,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:36,250 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:36,250 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:36,255 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53079
2016-05-17 21:52:36,256 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:36,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521956197_0001_000000, startTime=1463521956260
2016-05-17 21:52:36,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:36,260 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:36,323 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:36,323 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:36,323 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463521956197_0001, dagName=[19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36,343 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36,344 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521956197_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001/localmode-log-dir/dag_1463521956197_0001_1.dot
2016-05-17 21:52:36,345 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001/localmode-log-dir/dag_1463521956197_0001_1-tez-dag.pb.txt
2016-05-17 21:52:36,360 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521956197_0001_1
2016-05-17 21:52:36,360 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,361 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36 Running Dag: dag_1463521956197_0001_1
2016-05-17 21:52:36,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521956197_0001_1, submitTime=1463521956343
2016-05-17 21:52:36,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:36,381 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,381 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:36,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:36,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521956197_0001_1, initTime=1463521956380
2016-05-17 21:52:36,383 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956197_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:36,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_STARTED]: dagID=dag_1463521956197_0001_1, startTime=1463521956383
2016-05-17 21:52:36,383 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:36,384 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956197_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:36,384 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FF21EEC1C04942F492745C179A05836E : {F0420273451947A19C658D80B1A9CE89={InputName=F0420273451947A19C658D80B1A9CE89}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:36,384 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F0420273451947A19C658D80B1A9CE89, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:36,384 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,384 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,384 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,385 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]. Starting root input initializers: 1
2016-05-17 21:52:36,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:36,385 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:36,386 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,388 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F0420273451947A19C658D80B1A9CE89 on vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,390 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:36,390 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F0420273451947A19C658D80B1A9CE89 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:36,392 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:36,393 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463521956197_0001, dagName=[19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 21:52:36,393 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:36,394 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,394 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,394 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,394 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CB011DFD10FD4ED48B938E6E5F207A03, vertexId=vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:36,394 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CB011DFD10FD4ED48B938E6E5F207A03, vertexId=vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,398 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CB011DFD10FD4ED48B938E6E5F207A03, vertexName=A04EAE2F31B3429787C5735E432239E1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:36,399 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CB011DFD10FD4ED48B938E6E5F207A03, vertexId=vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:36,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A04EAE2F31B3429787C5735E432239E1, vertexId=vertex_1463521956197_0001_1_01, initRequestedTime=1463521956386, initedTime=1463521956401, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:36,402 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:36,404 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:36,407 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:36,410 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F0420273451947A19C658D80B1A9CE89 on vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,410 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:36,410 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F0420273451947A19C658D80B1A9CE89=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:36,411 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] parallelism set to 1
2016-05-17 21:52:36,411 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:36,412 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521956197_0001_1_00, eventCount=1
2016-05-17 21:52:36,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:36,414 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF21EEC1C04942F492745C179A05836E, vertexId=vertex_1463521956197_0001_1_00, initRequestedTime=1463521956384, initedTime=1463521956413, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:36,414 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] with distanceFromRoot: 0
2016-05-17 21:52:36,414 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:36,415 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,415 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:36,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521956197_0001_1_00, startRequestedTime=1463521956415, startedTime=1463521956415
2016-05-17 21:52:36,416 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:36,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:36,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_STARTED]: vertexName=FF21EEC1C04942F492745C179A05836E, taskId=task_1463521956197_0001_1_00_000000, scheduledTime=1463521956417, launchTime=1463521956417
2016-05-17 21:52:36,418 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:36,418 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521956197_0001_1_00 for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] numStartedSources: 1 numSources: 1
2016-05-17 21:52:36,419 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] with distanceFromRoot: 1
2016-05-17 21:52:36,419 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,420 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FF21EEC1C04942F492745C179A05836E in vertex: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,420 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A04EAE2F31B3429787C5735E432239E1 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:36,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:36,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521956197_0001_1_01, startRequestedTime=1463521956419, startedTime=1463521956419
2016-05-17 21:52:36,421 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:36,422 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult, VertexName: FF21EEC1C04942F492745C179A05836E, VertexParallelism: 1, TaskAttemptID:attempt_1463521956197_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0420273451947A19C658D80B1A9CE89, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A04EAE2F31B3429787C5735E432239E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:36,422 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:36,422 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:36,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521956197_0001, with dag id: dag_1463521956197_0001_1
2016-05-17 21:52:36,423 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:36,424 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:36,424 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:36,425 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521956197_0001_00_000001
2016-05-17 21:52:36,425 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:36,428 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521956197_0001_00_000001
2016-05-17 21:52:36,428 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:36,429 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:36,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:36,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521956197_0001_00_000001, launchTime=1463521956429
2016-05-17 21:52:36,431 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956197_0001_00_000001
2016-05-17 21:52:36,432 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521956197_0001_1_00_000000_0] to container: [container_1463521956197_0001_00_000001]
2016-05-17 21:52:36,432 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:36,433 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521956197_0001_1_00_000000_0] started. Is using containerId: [container_1463521956197_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:36,433 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:36,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF21EEC1C04942F492745C179A05836E, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0, startTime=1463521956432, containerId=container_1463521956197_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521956197_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:36,433 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:36,433 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:36,434 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521956197_0001_00_000001 given task: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,434 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956197_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36 Starting to run new task attempt: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,434 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,434 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,435 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:36,435 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:36,435 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:36,435 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:36,435 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:36,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult, VertexName: FF21EEC1C04942F492745C179A05836E, VertexParallelism: 1, TaskAttemptID:attempt_1463521956197_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0420273451947A19C658D80B1A9CE89, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A04EAE2F31B3429787C5735E432239E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:36,435 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:36,436 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:36,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,440 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521956197_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:36,440 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:36,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A04EAE2F31B3429787C5735E432239E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:36,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:36,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,447 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:36,447 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F0420273451947A19C658D80B1A9CE89, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:36,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:36,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,449 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,449 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF21EEC1C04942F492745C179A05836E, ordinal: 0
2016-05-17 21:52:36,450 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF21EEC1C04942F492745C179A05836E, mem on start (mb), free: 176, total: 554, max: 672
2016-05-17 21:52:36,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:36,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:36,453 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:36,454 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:36,455 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:36,455 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:36,455 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:36,455 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:36,455 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:36,456 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:36,456 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A04EAE2F31B3429787C5735E432239E1, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:36,456 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F0420273451947A19C658D80B1A9CE89, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:36,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:36,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,456 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:36,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F0420273451947A19C658D80B1A9CE89 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:36,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:36,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:36,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:36,458 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,459 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:36,461 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:36,461 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7e214d47
2016-05-17 21:52:36,462 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: F0420273451947A19C658D80B1A9CE89
2016-05-17 21:52:36,462 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 49A0EB06D5494EE09C0F1CC20E49081F
2016-05-17 21:52:36,463 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 49A0EB06D5494EE09C0F1CC20E49081F
2016-05-17 21:52:36,463 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7e214d47
2016-05-17 21:52:36,463 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:36,463 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:36,465 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@44f0adaf; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@151adf85; comparator=cascading.tuple.hadoop.util.TupleComparator@17489f96; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:36,465 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:36,465 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:36,466 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:36,479 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:36,479 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:36,479 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:36,479 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:36,479 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:36,555 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:36,556 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:36,556 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF21EEC1C04942F492745C179A05836E, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:36,559 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,559 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF21EEC1C04942F492745C179A05836E, mem on close (mb), free: 68, total: 554, max: 672
2016-05-17 21:52:36,559 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:36,559 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:36,560 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 22; bufvoid = 104857600
2016-05-17 21:52:36,560 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2016-05-17 21:52:36,563 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:36 Completed running task attempt: attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,566 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:36,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521956197_0001_1_00, eventCount=1
2016-05-17 21:52:36,567 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:36,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF21EEC1C04942F492745C179A05836E, taskAttemptId=attempt_1463521956197_0001_1_00_000000_0, startTime=1463521956432, finishTime=1463521956567, timeTaken=135, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956556, Process_Duration=3, Process_End_Time=1463521956559, Read_Duration=1, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:36,568 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:36,568 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:36,568 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521956197_0001_1_00_000000_0
2016-05-17 21:52:36,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:36,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_FINISHED]: vertexName=FF21EEC1C04942F492745C179A05836E, taskId=task_1463521956197_0001_1_00_000000, startTime=1463521956432, finishTime=1463521956568, timeTaken=136, status=SUCCEEDED, successfulAttemptID=attempt_1463521956197_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956556, Process_Duration=3, Process_End_Time=1463521956559, Read_Duration=1, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:36,569 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:36,569 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] : 1
2016-05-17 21:52:36,570 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,571 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:36,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_FINISHED]: vertexName=FF21EEC1C04942F492745C179A05836E, vertexId=vertex_1463521956197_0001_1_00, initRequestedTime=1463521956384, initedTime=1463521956413, startRequestedTime=1463521956415, startedTime=1463521956415, finishTime=1463521956570, timeTaken=155, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956556, Process_Duration=3, Process_End_Time=1463521956559, Read_Duration=1, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521956432, firstTasksToStart=[ task_1463521956197_0001_1_00_000000 ], lastTaskFinishTime=1463521956567, lastTasksToFinish=[ task_1463521956197_0001_1_00_000000 ], minTaskDuration=135, maxTaskDuration=135, avgTaskDuration=135.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521956197_0001_1_00_000000 ], longestDurationTasks=[ task_1463521956197_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:36,571 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:36,572 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:36,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521956556, Process_Duration=3, Process_End_Time=1463521956559, Read_Duration=1, Tuples_Read=5, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:36,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:36,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521956197_0001_00_000001, stoppedTime=1463521956572, exitStatus=0
2016-05-17 21:52:36,574 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:36,573 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:36,574 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,574 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] attempt: attempt_1463521956197_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:36,574 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956197_0001_00_000001
2016-05-17 21:52:36,575 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521956197_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:36,575 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956197_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:36,574 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A04EAE2F31B3429787C5735E432239E1
2016-05-17 21:52:36,575 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521956197_0001_00_000001, Exiting
2016-05-17 21:52:36,575 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521956197_0001_00_000001
2016-05-17 21:52:36,575 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521956197_0001_00_000001
2016-05-17 21:52:36,575 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,576 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:36,577 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521956197_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:36,578 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:36,578 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521956197_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:36,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_STARTED]: vertexName=A04EAE2F31B3429787C5735E432239E1, taskId=task_1463521956197_0001_1_01_000000, scheduledTime=1463521956578, launchTime=1463521956578
2016-05-17 21:52:36,578 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521956197_0001_00_000001 completed successfully
2016-05-17 21:52:36,578 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:36,579 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:36,579 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,579 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:36,579 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult, VertexName: A04EAE2F31B3429787C5735E432239E1, VertexParallelism: 1, TaskAttemptID:attempt_1463521956197_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF21EEC1C04942F492745C179A05836E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CB011DFD10FD4ED48B938E6E5F207A03, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:36,579 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:36,580 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:36,580 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521956197_0001_00_000002
2016-05-17 21:52:36,581 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:36,582 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521956197_0001_00_000002
2016-05-17 21:52:36,583 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:36,585 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:36,585 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:36,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521956197_0001_00_000002, launchTime=1463521956585
2016-05-17 21:52:36,586 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956197_0001_00_000002
2016-05-17 21:52:36,587 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521956197_0001_1_01_000000_0] to container: [container_1463521956197_0001_00_000002]
2016-05-17 21:52:36,587 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:36,588 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521956197_0001_00_000002 given task: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,588 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956197_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,588 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521956197_0001_1_01_000000_0] started. Is using containerId: [container_1463521956197_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:36,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A04EAE2F31B3429787C5735E432239E1, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0, startTime=1463521956588, containerId=container_1463521956197_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521956197_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:36,589 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:36,589 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:36,589 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:36 Starting to run new task attempt: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,590 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,591 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,591 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:36,591 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:36,592 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:36,592 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:36,592 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:36,592 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C] test/(1/1) ...SimpleResult/simpleresult, VertexName: A04EAE2F31B3429787C5735E432239E1, VertexParallelism: 1, TaskAttemptID:attempt_1463521956197_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF21EEC1C04942F492745C179A05836E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CB011DFD10FD4ED48B938E6E5F207A03, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:36,593 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:36,593 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:36,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,597 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521956197_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:36,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FF21EEC1C04942F492745C179A05836E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:36,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CB011DFD10FD4ED48B938E6E5F207A03, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:36,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:36,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CB011DFD10FD4ED48B938E6E5F207A03
2016-05-17 21:52:36,599 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:36,599 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CB011DFD10FD4ED48B938E6E5F207A03
2016-05-17 21:52:36,600 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:36,600 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,604 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:36,604 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:36,604 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,605 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:36,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,614 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:36,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CB011DFD10FD4ED48B938E6E5F207A03
2016-05-17 21:52:36,616 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A04EAE2F31B3429787C5735E432239E1, ordinal: 1
2016-05-17 21:52:36,616 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A04EAE2F31B3429787C5735E432239E1, mem on start (mb), free: 62, total: 554, max: 672
2016-05-17 21:52:36,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:36,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:36,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:36,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:36,616 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:36,618 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:36,618 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:36,618 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:36,618 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:36,618 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:36,619 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CB011DFD10FD4ED48B938E6E5F207A03, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:36,619 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FF21EEC1C04942F492745C179A05836E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:36,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:36,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FF21EEC1C04942F492745C179A05836E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:36,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:36,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:36,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,621 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:36,622 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:36,622 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FF21EEC1C04942F492745C179A05836E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:36,622 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:36,622 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:36,622 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:36,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:36,625 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,625 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521956197_0001_1_00_000000_0_10039, runDuration: 122339, ]
2016-05-17 21:52:36,626 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521956197_0001_1_00_000000_0_10039] to DISK_DIRECT, CompressedSize=36, DecompressedSize=32,EndTime=1463521956626, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:52:36,627 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,627 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:36,627 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [FF21EEC1C04942F492745C179A05836E] #7 in 1ms
2016-05-17 21:52:36,627 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FF21EEC1C04942F492745C179A05836E] #7, status:false, isInterrupted:false
2016-05-17 21:52:36,628 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:36,628 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 36 bytes from disk
2016-05-17 21:52:36,628 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:36,628 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:36,630 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2016-05-17 21:52:36,630 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,631 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:36,630 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:36,632 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 49A0EB06D5494EE09C0F1CC20E49081F
2016-05-17 21:52:36,632 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"], id: CB011DFD10FD4ED48B938E6E5F207A03
2016-05-17 21:52:36,633 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-17 21:52:36,634 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 49A0EB06D5494EE09C0F1CC20E49081F, for 1 inputs
2016-05-17 21:52:36,634 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A04EAE2F31B3429787C5735E432239E1, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:36,638 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@56f47006; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@49e6c683
2016-05-17 21:52:36,639 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:36,640 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:36,640 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,641 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521956197_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:36,642 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,643 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A04EAE2F31B3429787C5735E432239E1, mem on close (mb), free: 58, total: 554, max: 672
2016-05-17 21:52:36,643 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FF21EEC1C04942F492745C179A05836E
2016-05-17 21:52:36,644 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:36,645 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:36,645 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:36,645 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:36,646 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:36 Completed running task attempt: attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,648 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:36,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521956634, Process_Duration=8, Process_End_Time=1463521956642, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=2]]
2016-05-17 21:52:36,649 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:36,650 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:36,650 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,650 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956197_0001_00_000002
2016-05-17 21:52:36,649 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A04EAE2F31B3429787C5735E432239E1, taskAttemptId=attempt_1463521956197_0001_1_01_000000_0, startTime=1463521956588, finishTime=1463521956649, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956634, Process_Duration=8, Process_End_Time=1463521956642, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-05-17 21:52:36,650 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956197_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:36,650 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:36,651 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521956197_0001_1_01_000000_0
2016-05-17 21:52:36,651 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:36,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:TASK_FINISHED]: vertexName=A04EAE2F31B3429787C5735E432239E1, taskId=task_1463521956197_0001_1_01_000000, startTime=1463521956588, finishTime=1463521956651, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463521956197_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956634, Process_Duration=8, Process_End_Time=1463521956642, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-05-17 21:52:36,652 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956197_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:36,652 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] : 1
2016-05-17 21:52:36,652 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,653 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:36,653 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:VERTEX_FINISHED]: vertexName=A04EAE2F31B3429787C5735E432239E1, vertexId=vertex_1463521956197_0001_1_01, initRequestedTime=1463521956386, initedTime=1463521956401, startRequestedTime=1463521956419, startedTime=1463521956419, finishTime=1463521956652, timeTaken=233, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521956634, Process_Duration=8, Process_End_Time=1463521956642, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2, vertexStats=firstTaskStartTime=1463521956588, firstTasksToStart=[ task_1463521956197_0001_1_01_000000 ], lastTaskFinishTime=1463521956649, lastTasksToFinish=[ task_1463521956197_0001_1_01_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521956197_0001_1_01_000000 ], longestDurationTasks=[ task_1463521956197_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:36,653 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:36,654 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:36,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521956197_0001_00_000002, stoppedTime=1463521956654, exitStatus=0
2016-05-17 21:52:36,654 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:36,654 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:36,654 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:36,654 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521956197_0001_1
2016-05-17 21:52:36,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521956197_0001_1
2016-05-17 21:52:36,656 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CB011DFD10FD4ED48B938E6E5F207A03 for vertex: vertex_1463521956197_0001_1_01 [A04EAE2F31B3429787C5735E432239E1]
2016-05-17 21:52:36,660 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521956197_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:36,660 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956197_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:36,660 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521956197_0001_00_000002, Exiting
2016-05-17 21:52:36,661 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521956197_0001_00_000002
2016-05-17 21:52:36,661 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521956197_0001_00_000002
2016-05-17 21:52:36,662 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521956197_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:36,662 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521956197_0001_00_000002 completed successfully
2016-05-17 21:52:36,664 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521956197_0001_1_00 [FF21EEC1C04942F492745C179A05836E]
2016-05-17 21:52:36,667 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521956197_0001_1, queueSize=0
2016-05-17 21:52:36,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:36,668 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956197_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521956197_0001_1, startTime=1463521956383, finishTime=1463521956665, timeTaken=282, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=130, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1162870784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043913190, Process_Duration=11, Process_End_Time=2927043913201, Read_Duration=1, Tuples_Read=7, Tuples_Written=4, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=2
2016-05-17 21:52:36,668 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521956197_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:36,668 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956197_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:36,668 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956197_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:36,669 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521956197_0001_00_000002
2016-05-17 21:52:36 Completed Dag: dag_1463521956197_0001_1
2016-05-17 21:52:36,670 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521956197_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:36,670 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521956197_0001_1_post
2016-05-17 21:52:36,670 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:36,671 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:36,676 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463521956197_0001
2016-05-17 21:52:36,678 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:36,678 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:36,679 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:36,679 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:36,679 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:36,679 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:36,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:36,680 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:36,680 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:36,680 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:36,681 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:36,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:36,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:36,686 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/19FD1350FDBC414DB3D49A0F066F4599/C30EAE1A4FC346AB89DF8D5E41A81B9C/.tez/application_1463521956197_0001
2016-05-17 21:52:36,687 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult/_temporary
2016-05-17 21:52:36,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.519
2016-05-17 21:52:36,723 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-17 21:52:36,758 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:36,758 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:36,796 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:52:36,804 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 21:52:36,805 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:36,805 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:36,806 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:36,822 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:36,824 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-17 21:52:36,834 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:36,834 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:36,834 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-17 21:52:36,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:36,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:36,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:36,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:36,837 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:36,843 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:36,843 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:36,856 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:36,861 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:36,861 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:36,861 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:36,872 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001 doesn't exist and is created
2016-05-17 21:52:36,889 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:36,890 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:36,890 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001
2016-05-17 21:52:36,893 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521956861_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:36,903 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:36,904 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:36,906 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:36,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:36,906 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001/localmode-log-dir
2016-05-17 21:52:36,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001/localmode-log-dir/history.txt.appattempt_1463521956861_0001_000000, maxErrors=10
2016-05-17 21:52:36,907 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:36,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521956861_0001_000000, appSubmitTime=1463521956893, launchTime=1463521956893
2016-05-17 21:52:36,913 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36648
2016-05-17 21:52:36,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:36,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:36,917 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:36,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:36,919 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:36,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521956861_0001_000000, startTime=1463521956921
2016-05-17 21:52:36,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:36,921 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:36,990 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:36,990 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:36,990 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463521956861_0001, dagName=[66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:37,006 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:37,008 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521956861_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001/localmode-log-dir/dag_1463521956861_0001_1.dot
2016-05-17 21:52:37,008 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001/localmode-log-dir/dag_1463521956861_0001_1-tez-dag.pb.txt
2016-05-17 21:52:37,017 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521956861_0001_1
2016-05-17 21:52:37,017 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,017 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:37 Running Dag: dag_1463521956861_0001_1
2016-05-17 21:52:37,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521956861_0001_1, submitTime=1463521957006
2016-05-17 21:52:37,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:37,030 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D4A883C13A1A4646A28E43F824B680AB
2016-05-17 21:52:37,030 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:37,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521956861_0001_1, initTime=1463521957030
2016-05-17 21:52:37,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:37,032 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956861_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:37,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_STARTED]: dagID=dag_1463521956861_0001_1, startTime=1463521957032
2016-05-17 21:52:37,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:37,033 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:37,033 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956861_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:37,033 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D4A883C13A1A4646A28E43F824B680AB : {372E46651972455886F26311FE2E210F={InputName=372E46651972455886F26311FE2E210F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:37,033 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 372E46651972455886F26311FE2E210F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:37,033 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,034 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,034 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,034 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463521956861_0001, dagName=[66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 21:52:37,034 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]. Starting root input initializers: 1
2016-05-17 21:52:37,035 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:37,035 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 372E46651972455886F26311FE2E210F on vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,037 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:37,037 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 372E46651972455886F26311FE2E210F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:37,038 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:37,038 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:37,040 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:37,041 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 372E46651972455886F26311FE2E210F on vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,041 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:37,041 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {372E46651972455886F26311FE2E210F=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:37,042 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] parallelism set to 1
2016-05-17 21:52:37,042 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:37,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521956861_0001_1_00, eventCount=1
2016-05-17 21:52:37,042 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,042 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A3CAE8CF934D4F518F3AA3431C98869A, vertexId=vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:37,043 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A3CAE8CF934D4F518F3AA3431C98869A, vertexId=vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,044 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A3CAE8CF934D4F518F3AA3431C98869A, vertexName=D4A883C13A1A4646A28E43F824B680AB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:37,045 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A3CAE8CF934D4F518F3AA3431C98869A, vertexId=vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:37,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, vertexId=vertex_1463521956861_0001_1_00, initRequestedTime=1463521957033, initedTime=1463521957046, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:37,047 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] with distanceFromRoot: 0
2016-05-17 21:52:37,047 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:37,047 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D4A883C13A1A4646A28E43F824B680AB
2016-05-17 21:52:37,047 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:37,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521956861_0001_1_00, startRequestedTime=1463521957047, startedTime=1463521957047
2016-05-17 21:52:37,048 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:37,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:37,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:TASK_STARTED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, taskId=task_1463521956861_0001_1_00_000000, scheduledTime=1463521957048, launchTime=1463521957048
2016-05-17 21:52:37,049 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956861_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:37,049 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,049 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall, VertexName: D4A883C13A1A4646A28E43F824B680AB, VertexParallelism: 1, TaskAttemptID:attempt_1463521956861_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=372E46651972455886F26311FE2E210F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A3CAE8CF934D4F518F3AA3431C98869A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:37,050 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956861_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:37,050 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:37,050 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:37,051 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:37,051 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:37,052 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521956861_0001_00_000001
2016-05-17 21:52:37,052 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:37,053 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521956861_0001_00_000001
2016-05-17 21:52:37,053 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:37,054 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:37,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:37,055 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521956861_0001, with dag id: dag_1463521956861_0001_1
2016-05-17 21:52:37,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521956861_0001_00_000001, launchTime=1463521957054
2016-05-17 21:52:37,056 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956861_0001_00_000001
2016-05-17 21:52:37,058 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521956861_0001_1_00_000000_0] to container: [container_1463521956861_0001_00_000001]
2016-05-17 21:52:37,058 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:37,060 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521956861_0001_1_00_000000_0] started. Is using containerId: [container_1463521956861_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:37,061 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0, startTime=1463521957059, containerId=container_1463521956861_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521956861_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:37,061 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956861_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:37,061 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956861_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:37,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:37,062 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521956861_0001_00_000001 given task: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,062 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956861_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37 Starting to run new task attempt: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,063 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,063 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,063 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:37,063 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:37,063 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:37,063 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:37,063 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:37,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3] test/(1/1) ...m/testFilterAll/filterall, VertexName: D4A883C13A1A4646A28E43F824B680AB, VertexParallelism: 1, TaskAttemptID:attempt_1463521956861_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=372E46651972455886F26311FE2E210F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A3CAE8CF934D4F518F3AA3431C98869A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:37,064 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:37,064 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:37,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,067 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521956861_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:37,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=372E46651972455886F26311FE2E210F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:37,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:37,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,069 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A3CAE8CF934D4F518F3AA3431C98869A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:37,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:37,070 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:37,072 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:37,073 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A3CAE8CF934D4F518F3AA3431C98869A
2016-05-17 21:52:37,074 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:37,074 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A3CAE8CF934D4F518F3AA3431C98869A
2016-05-17 21:52:37,083 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D4A883C13A1A4646A28E43F824B680AB, ordinal: 0
2016-05-17 21:52:37,083 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4A883C13A1A4646A28E43F824B680AB, mem on start (mb), free: 204, total: 554, max: 672
2016-05-17 21:52:37,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:37,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:37,087 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:37,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A3CAE8CF934D4F518F3AA3431C98869A
2016-05-17 21:52:37,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:37,088 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:37,089 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:37,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:37,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:37,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:37,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:37,091 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:37,091 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 372E46651972455886F26311FE2E210F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:37,091 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A3CAE8CF934D4F518F3AA3431C98869A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:37,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:37,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 372E46651972455886F26311FE2E210F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:37,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,092 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:37,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:37,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:37,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:37,093 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,096 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:37,096 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 372E46651972455886F26311FE2E210F
2016-05-17 21:52:37,096 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:37,096 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"], id: A3CAE8CF934D4F518F3AA3431C98869A
2016-05-17 21:52:37,097 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-17 21:52:37,097 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:37,097 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:37,097 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7e5bf9e7
2016-05-17 21:52:37,099 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7e5bf9e7
2016-05-17 21:52:37,099 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:37,100 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:37,100 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D4A883C13A1A4646A28E43F824B680AB, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:37,102 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:37,102 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:37,102 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,102 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521956861_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:37,103 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,103 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4A883C13A1A4646A28E43F824B680AB, mem on close (mb), free: 203, total: 554, max: 672
2016-05-17 21:52:37 Completed running task attempt: attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,104 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:37,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, taskAttemptId=attempt_1463521956861_0001_1_00_000000_0, startTime=1463521957059, finishTime=1463521957104, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:37,105 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521956861_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:37,105 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521956861_0001_1_00_000000_0
2016-05-17 21:52:37,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:TASK_FINISHED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, taskId=task_1463521956861_0001_1_00_000000, startTime=1463521957059, finishTime=1463521957105, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463521956861_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:37,106 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521956861_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:37,106 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] : 1
2016-05-17 21:52:37,106 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:37,107 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:37,107 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:37,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:VERTEX_FINISHED]: vertexName=D4A883C13A1A4646A28E43F824B680AB, vertexId=vertex_1463521956861_0001_1_00, initRequestedTime=1463521957033, initedTime=1463521957046, startRequestedTime=1463521957047, startedTime=1463521957047, finishTime=1463521957106, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463521957059, firstTasksToStart=[ task_1463521956861_0001_1_00_000000 ], lastTaskFinishTime=1463521957104, lastTasksToFinish=[ task_1463521956861_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521956861_0001_1_00_000000 ], longestDurationTasks=[ task_1463521956861_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:37,108 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:37,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:37,109 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:37,109 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:37,109 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521956861_0001_1
2016-05-17 21:52:37,109 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521956861_0001_1
2016-05-17 21:52:37,109 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A3CAE8CF934D4F518F3AA3431C98869A for vertex: vertex_1463521956861_0001_1_00 [D4A883C13A1A4646A28E43F824B680AB]
2016-05-17 21:52:37,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 21:52:37,112 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:37,113 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,113 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521956861_0001_00_000001
2016-05-17 21:52:37,117 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521956861_0001_1, queueSize=0
2016-05-17 21:52:37,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:37,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521956861_0001_1, startTime=1463521957032, finishTime=1463521957116, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957100, Process_Duration=3, Process_End_Time=1463521957103, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:37,118 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521956861_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:37,119 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521956861_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:37,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:37,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521956861_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521956861_0001_00_000001, stoppedTime=1463521957119, exitStatus=0
2016-05-17 21:52:37,120 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:37 Completed Dag: dag_1463521956861_0001_1
2016-05-17 21:52:37,121 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521956861_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:37,121 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521956861_0001_1_post
2016-05-17 21:52:37,121 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,122 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:37,122 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521956861_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:37,122 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:37,123 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521956861_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:37,123 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521956861_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:37,123 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521956861_0001_00_000001, Exiting
2016-05-17 21:52:37,124 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521956861_0001_00_000001
2016-05-17 21:52:37,124 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521956861_0001_00_000001
2016-05-17 21:52:37,127 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521956861_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:37,127 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521956861_0001_00_000001 completed successfully
2016-05-17 21:52:37,128 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521956861_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:37,127 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463521956861_0001
2016-05-17 21:52:37,130 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:37,130 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:37,130 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:37,130 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:37,130 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:37,131 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:37,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:37,131 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:37,131 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:37,131 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:37,131 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:37,132 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:37,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:37,138 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66E569E0CD96463B8FF5FFB65B8F8C08/BA121A8AEDE048E880872DF015EE13A3/.tez/application_1463521956861_0001
2016-05-17 21:52:37,138 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall/_temporary
2016-05-17 21:52:37,139 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.305
2016-05-17 21:52:37,175 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-17 21:52:37,204 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:37,205 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:37,247 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 21:52:37,260 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 21:52:37,260 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:37,261 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:37,262 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:37,277 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 21:52:37,285 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-17 21:52:37,336 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:37,336 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:37,336 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-17 21:52:37,336 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:37,336 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:37,336 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:37,337 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37,337 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:37,350 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:37,354 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:37,355 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:37,355 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:37,364 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001 doesn't exist and is created
2016-05-17 21:52:37,380 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:37,381 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:37,381 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001
2016-05-17 21:52:37,386 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521957355_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:37,396 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:37,397 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:37,400 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:37,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:37,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001/localmode-log-dir
2016-05-17 21:52:37,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001/localmode-log-dir/history.txt.appattempt_1463521957355_0001_000000, maxErrors=10
2016-05-17 21:52:37,402 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:37,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521957355_0001_000000, appSubmitTime=1463521957385, launchTime=1463521957385
2016-05-17 21:52:37,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:37,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:37,410 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58009
2016-05-17 21:52:37,412 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:37,412 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:37,413 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:37,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521957355_0001_000000, startTime=1463521957417
2016-05-17 21:52:37,417 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:37,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:37,481 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:37,481 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:37,481 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463521957355_0001, dagName=[41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37,498 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37,499 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521957355_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001/localmode-log-dir/dag_1463521957355_0001_1.dot
2016-05-17 21:52:37,500 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001/localmode-log-dir/dag_1463521957355_0001_1-tez-dag.pb.txt
2016-05-17 21:52:37,513 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521957355_0001_1
2016-05-17 21:52:37,513 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,513 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37 Running Dag: dag_1463521957355_0001_1
2016-05-17 21:52:37,526 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521957355_0001_1, submitTime=1463521957498
2016-05-17 21:52:37,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:37,528 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,528 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:37,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521957355_0001_1, initTime=1463521957526
2016-05-17 21:52:37,529 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521957355_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:37,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:37,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_STARTED]: dagID=dag_1463521957355_0001_1, startTime=1463521957529
2016-05-17 21:52:37,530 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:37,531 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521957355_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:37,531 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FD7472811FE94572A8A815EDDB0A8842 : {C66A7A28B69B4DEF8D8CE1EB85C45DBE={InputName=C66A7A28B69B4DEF8D8CE1EB85C45DBE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:37,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:37,531 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463521957355_0001, dagName=[41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown
2016-05-17 21:52:37,531 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C66A7A28B69B4DEF8D8CE1EB85C45DBE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:37,532 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,532 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,532 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,532 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]. Starting root input initializers: 1
2016-05-17 21:52:37,532 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:37,533 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,534 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C66A7A28B69B4DEF8D8CE1EB85C45DBE on vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,535 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:37,536 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:37,536 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,536 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,537 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,537 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=15C86E44C79B4390A54B7D031375A2B9, vertexId=vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:37,537 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=15C86E44C79B4390A54B7D031375A2B9, vertexId=vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,536 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C66A7A28B69B4DEF8D8CE1EB85C45DBE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:37,539 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:37,539 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:37,540 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=15C86E44C79B4390A54B7D031375A2B9, vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:37,540 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=15C86E44C79B4390A54B7D031375A2B9, vertexId=vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,541 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:37,542 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C66A7A28B69B4DEF8D8CE1EB85C45DBE on vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:37,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, vertexId=vertex_1463521957355_0001_1_01, initRequestedTime=1463521957533, initedTime=1463521957542, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:37,543 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:37,543 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:37,543 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C66A7A28B69B4DEF8D8CE1EB85C45DBE=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:37,543 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] parallelism set to 1
2016-05-17 21:52:37,544 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:37,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521957355_0001_1_00, eventCount=1
2016-05-17 21:52:37,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, vertexId=vertex_1463521957355_0001_1_00, initRequestedTime=1463521957531, initedTime=1463521957544, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:37,545 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] with distanceFromRoot: 0
2016-05-17 21:52:37,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:37,545 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:37,545 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,546 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:37,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521957355_0001_1_00, startRequestedTime=1463521957545, startedTime=1463521957545
2016-05-17 21:52:37,547 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:37,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:37,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_STARTED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, taskId=task_1463521957355_0001_1_00_000000, scheduledTime=1463521957547, launchTime=1463521957547
2016-05-17 21:52:37,548 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:37,548 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521957355_0001_1_00 for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] numStartedSources: 1 numSources: 1
2016-05-17 21:52:37,548 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] with distanceFromRoot: 1
2016-05-17 21:52:37,549 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,549 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FD7472811FE94572A8A815EDDB0A8842 in vertex: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,549 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E4FB3BD594E64AE2AAF803F99DE6B0E7 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:37,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:37,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521957355_0001_1_01, startRequestedTime=1463521957549, startedTime=1463521957549
2016-05-17 21:52:37,550 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:37,551 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown, VertexName: FD7472811FE94572A8A815EDDB0A8842, VertexParallelism: 1, TaskAttemptID:attempt_1463521957355_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C66A7A28B69B4DEF8D8CE1EB85C45DBE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:37,551 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:37,551 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:37,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521957355_0001, with dag id: dag_1463521957355_0001_1
2016-05-17 21:52:37,553 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:37,554 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:37,555 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:37,556 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521957355_0001_00_000001
2016-05-17 21:52:37,556 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:37,558 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521957355_0001_00_000001
2016-05-17 21:52:37,558 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:37,559 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:37,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:37,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521957355_0001_00_000001, launchTime=1463521957559
2016-05-17 21:52:37,560 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521957355_0001_00_000001
2016-05-17 21:52:37,561 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521957355_0001_1_00_000000_0] to container: [container_1463521957355_0001_00_000001]
2016-05-17 21:52:37,561 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:37,562 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521957355_0001_1_00_000000_0] started. Is using containerId: [container_1463521957355_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:37,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0, startTime=1463521957561, containerId=container_1463521957355_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521957355_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:37,562 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:37,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:37,562 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:37,562 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521957355_0001_00_000001 given task: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,562 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521957355_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37 Starting to run new task attempt: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,563 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,563 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,563 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:37,563 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:37,563 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:37,563 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:37,564 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:37,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown, VertexName: FD7472811FE94572A8A815EDDB0A8842, VertexParallelism: 1, TaskAttemptID:attempt_1463521957355_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C66A7A28B69B4DEF8D8CE1EB85C45DBE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:37,564 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:37,564 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:37,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,565 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521957355_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:37,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C66A7A28B69B4DEF8D8CE1EB85C45DBE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:37,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:37,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,566 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,569 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:37,569 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,569 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:37,569 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:37,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:37,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,573 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:37,573 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,575 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FD7472811FE94572A8A815EDDB0A8842, ordinal: 0
2016-05-17 21:52:37,575 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD7472811FE94572A8A815EDDB0A8842, mem on start (mb), free: 116, total: 554, max: 672
2016-05-17 21:52:37,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:37,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:37,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:37,577 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:37,577 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:37,578 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:37,578 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:37,578 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:37,578 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:37,579 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C66A7A28B69B4DEF8D8CE1EB85C45DBE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:37,579 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E4FB3BD594E64AE2AAF803F99DE6B0E7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:37,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:37,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C66A7A28B69B4DEF8D8CE1EB85C45DBE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:37,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:37,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:37,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,581 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:37,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:37,582 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,585 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:37,585 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: C66A7A28B69B4DEF8D8CE1EB85C45DBE
2016-05-17 21:52:37,586 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:-1]], id: ABD8A56FF531456D8D4382DA26C94D4E
2016-05-17 21:52:37,586 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:-1]] ABD8A56FF531456D8D4382DA26C94D4E
2016-05-17 21:52:37,588 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@9988946; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@7d5c5795; comparator=cascading.tuple.hadoop.util.TupleComparator@6d19054; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:37,589 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:37,589 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:37,590 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:37,604 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:37,605 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5c167eac
2016-05-17 21:52:37,607 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5c167eac
2016-05-17 21:52:37,607 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:37,607 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:37,618 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:37,618 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:37,619 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:37,619 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:37,619 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:37,619 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:37,620 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:37,620 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FD7472811FE94572A8A815EDDB0A8842, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:37,622 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,623 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD7472811FE94572A8A815EDDB0A8842, mem on close (mb), free: 158, total: 554, max: 672
2016-05-17 21:52:37,623 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:37,623 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:37,623 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:37,623 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:37,626 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:37 Completed running task attempt: attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,630 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:37,631 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521957620, Process_Duration=2, Process_End_Time=1463521957622, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:37,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521957355_0001_1_00, eventCount=1
2016-05-17 21:52:37,631 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:37,631 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,632 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521957355_0001_00_000001
2016-05-17 21:52:37,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, taskAttemptId=attempt_1463521957355_0001_1_00_000000_0, startTime=1463521957561, finishTime=1463521957632, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957620, Process_Duration=2, Process_End_Time=1463521957622, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:37,632 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:37,632 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:37,632 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521957355_0001_1_00_000000_0
2016-05-17 21:52:37,632 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:37,633 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:37,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_FINISHED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, taskId=task_1463521957355_0001_1_00_000000, startTime=1463521957561, finishTime=1463521957633, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1463521957355_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957620, Process_Duration=2, Process_End_Time=1463521957622, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:37,634 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:37,634 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] : 1
2016-05-17 21:52:37,634 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,635 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:37,635 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_FINISHED]: vertexName=FD7472811FE94572A8A815EDDB0A8842, vertexId=vertex_1463521957355_0001_1_00, initRequestedTime=1463521957531, initedTime=1463521957544, startRequestedTime=1463521957545, startedTime=1463521957545, finishTime=1463521957634, timeTaken=89, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957620, Process_Duration=2, Process_End_Time=1463521957622, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521957561, firstTasksToStart=[ task_1463521957355_0001_1_00_000000 ], lastTaskFinishTime=1463521957632, lastTasksToFinish=[ task_1463521957355_0001_1_00_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521957355_0001_1_00_000000 ], longestDurationTasks=[ task_1463521957355_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:37,636 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:37,636 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:37,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521957355_0001_00_000001, stoppedTime=1463521957636, exitStatus=0
2016-05-17 21:52:37,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:37,636 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:37,636 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] attempt: attempt_1463521957355_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:37,637 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E4FB3BD594E64AE2AAF803F99DE6B0E7
2016-05-17 21:52:37,637 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,637 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:37,637 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521957355_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:37,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_STARTED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, taskId=task_1463521957355_0001_1_01_000000, scheduledTime=1463521957638, launchTime=1463521957638
2016-05-17 21:52:37,638 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:37,639 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:37,639 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,639 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown, VertexName: E4FB3BD594E64AE2AAF803F99DE6B0E7, VertexParallelism: 1, TaskAttemptID:attempt_1463521957355_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FD7472811FE94572A8A815EDDB0A8842, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=15C86E44C79B4390A54B7D031375A2B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:37,639 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:37,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:37,640 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:37,641 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521957355_0001_00_000002
2016-05-17 21:52:37,641 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:37,643 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521957355_0001_00_000002
2016-05-17 21:52:37,643 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:37,644 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:37,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:37,645 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521957355_0001_00_000002, launchTime=1463521957644
2016-05-17 21:52:37,645 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521957355_0001_00_000002
2016-05-17 21:52:37,646 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521957355_0001_1_01_000000_0] to container: [container_1463521957355_0001_00_000002]
2016-05-17 21:52:37,647 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:37,647 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521957355_0001_1_01_000000_0] started. Is using containerId: [container_1463521957355_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:37,648 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0, startTime=1463521957647, containerId=container_1463521957355_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521957355_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:37,648 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:37,648 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:37,648 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:37,649 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521957355_0001_00_000002 given task: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,649 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521957355_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37 Starting to run new task attempt: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,650 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,650 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,650 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:37,650 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:37,650 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521957355_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:37,650 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:37,651 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521957355_0001_00_000001: 19 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:37,651 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521957355_0001_00_000001, Exiting
2016-05-17 21:52:37,651 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521957355_0001_00_000001
2016-05-17 21:52:37,651 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521957355_0001_00_000001
2016-05-17 21:52:37,651 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:37,662 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:37,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88] test/(1/1) ...own/simplerelativeunknown, VertexName: E4FB3BD594E64AE2AAF803F99DE6B0E7, VertexParallelism: 1, TaskAttemptID:attempt_1463521957355_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FD7472811FE94572A8A815EDDB0A8842, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=15C86E44C79B4390A54B7D031375A2B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:37,662 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:37,664 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,664 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:37,664 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,665 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521957355_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:37,666 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521957355_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:37,667 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521957355_0001_00_000001 completed successfully
2016-05-17 21:52:37,667 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:37,668 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FD7472811FE94572A8A815EDDB0A8842, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:37,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:37,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,671 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:37,672 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:37,672 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=15C86E44C79B4390A54B7D031375A2B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:37,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:37,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 15C86E44C79B4390A54B7D031375A2B9
2016-05-17 21:52:37,674 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:37,674 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 15C86E44C79B4390A54B7D031375A2B9
2016-05-17 21:52:37,676 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:37,690 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:37,690 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 15C86E44C79B4390A54B7D031375A2B9
2016-05-17 21:52:37,691 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4FB3BD594E64AE2AAF803F99DE6B0E7, ordinal: 1
2016-05-17 21:52:37,691 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4FB3BD594E64AE2AAF803F99DE6B0E7, mem on start (mb), free: 150, total: 554, max: 672
2016-05-17 21:52:37,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:37,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:37,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:37,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:37,692 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:37,692 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:37,693 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:37,693 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:37,693 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:37,694 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:37,694 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FD7472811FE94572A8A815EDDB0A8842, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:37,694 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 15C86E44C79B4390A54B7D031375A2B9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:37,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:37,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FD7472811FE94572A8A815EDDB0A8842 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:37,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:37,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:37,695 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:37,696 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:37,696 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FD7472811FE94572A8A815EDDB0A8842 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:37,696 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:37,696 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:37,697 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:37,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:37,698 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,699 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521957355_0001_1_00_000000_0_10049, runDuration: 57039, ]
2016-05-17 21:52:37,701 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:-1]]
2016-05-17 21:52:37,701 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521957355_0001_1_00_000000_0_10049] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521957701, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 21:52:37,701 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:-1]] streamed: true, id: ABD8A56FF531456D8D4382DA26C94D4E
2016-05-17 21:52:37,702 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"], id: 15C86E44C79B4390A54B7D031375A2B9
2016-05-17 21:52:37,702 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-17 21:52:37,702 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:-1]] ABD8A56FF531456D8D4382DA26C94D4E, for 1 inputs
2016-05-17 21:52:37,701 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,703 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:37,703 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [FD7472811FE94572A8A815EDDB0A8842] #8 in 3ms
2016-05-17 21:52:37,704 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FD7472811FE94572A8A815EDDB0A8842] #8, status:false, isInterrupted:false
2016-05-17 21:52:37,704 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:37,705 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:37,705 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:37,705 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:37,706 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:37,706 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,706 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:37,706 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4FB3BD594E64AE2AAF803F99DE6B0E7, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:52:37,707 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@12b89aea; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@15c28240
2016-05-17 21:52:37,709 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:37,709 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:37,709 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,710 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521957355_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:37,710 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,710 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4FB3BD594E64AE2AAF803F99DE6B0E7, mem on close (mb), free: 147, total: 554, max: 672
2016-05-17 21:52:37,710 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FD7472811FE94572A8A815EDDB0A8842
2016-05-17 21:52:37,711 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:37,711 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:37,711 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:37,711 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:37,712 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:37 Completed running task attempt: attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,712 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:37,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521957706, Process_Duration=4, Process_End_Time=1463521957710, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 21:52:37,713 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:37,713 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, taskAttemptId=attempt_1463521957355_0001_1_01_000000_0, startTime=1463521957647, finishTime=1463521957713, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957706, Process_Duration=4, Process_End_Time=1463521957710, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:37,714 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521957355_0001_00_000002
2016-05-17 21:52:37,714 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521957355_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:37,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:37,714 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521957355_0001_1_01_000000_0
2016-05-17 21:52:37,714 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:37,715 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:37,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:TASK_FINISHED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, taskId=task_1463521957355_0001_1_01_000000, startTime=1463521957647, finishTime=1463521957714, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1463521957355_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957706, Process_Duration=4, Process_End_Time=1463521957710, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:37,715 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521957355_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:37,715 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] : 1
2016-05-17 21:52:37,715 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,716 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:37,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:VERTEX_FINISHED]: vertexName=E4FB3BD594E64AE2AAF803F99DE6B0E7, vertexId=vertex_1463521957355_0001_1_01, initRequestedTime=1463521957533, initedTime=1463521957542, startRequestedTime=1463521957549, startedTime=1463521957549, finishTime=1463521957715, timeTaken=166, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581435392, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521957706, Process_Duration=4, Process_End_Time=1463521957710, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463521957647, firstTasksToStart=[ task_1463521957355_0001_1_01_000000 ], lastTaskFinishTime=1463521957713, lastTasksToFinish=[ task_1463521957355_0001_1_01_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521957355_0001_1_01_000000 ], longestDurationTasks=[ task_1463521957355_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:37,717 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:37,717 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:37,717 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:37,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521957355_0001_00_000002, stoppedTime=1463521957717, exitStatus=0
2016-05-17 21:52:37,718 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:37,718 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:37,718 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521957355_0001_1
2016-05-17 21:52:37,719 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521957355_0001_1
2016-05-17 21:52:37,719 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521957355_0001_1_00 [FD7472811FE94572A8A815EDDB0A8842]
2016-05-17 21:52:37,719 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 15C86E44C79B4390A54B7D031375A2B9 for vertex: vertex_1463521957355_0001_1_01 [E4FB3BD594E64AE2AAF803F99DE6B0E7]
2016-05-17 21:52:37,724 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521957355_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:37,724 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521957355_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:37,724 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521957355_0001_00_000002, Exiting
2016-05-17 21:52:37,724 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521957355_0001_00_000002
2016-05-17 21:52:37,725 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521957355_0001_00_000002
2016-05-17 21:52:37,725 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521957355_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:37,725 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521957355_0001_00_000002 completed successfully
2016-05-17 21:52:37,729 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521957355_0001_1, queueSize=0
2016-05-17 21:52:37,729 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:37,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521957355_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521957355_0001_1, startTime=1463521957529, finishTime=1463521957727, timeTaken=198, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=189, FILE_BYTES_WRITTEN=159, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1162870784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043915326, Process_Duration=6, Process_End_Time=2927043915332, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-17 21:52:37,730 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521957355_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:37,730 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521957355_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:37,730 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521957355_0001_00_000002
2016-05-17 21:52:37,730 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521957355_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:37 Completed Dag: dag_1463521957355_0001_1
2016-05-17 21:52:37,731 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521957355_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:37,731 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521957355_0001_1_post
2016-05-17 21:52:37,731 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:37,731 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:37,737 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463521957355_0001
2016-05-17 21:52:37,739 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:37,739 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:37,739 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:37,739 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:37,739 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:37,740 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:37,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:37,741 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:37,741 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:37,741 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:37,741 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:37,742 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:37,742 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:37,747 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/41FCE1F34DDC474A98385F981250FA32/A5BFF1C228764F8986A47F9F8DFE1E88/.tez/application_1463521957355_0001
2016-05-17 21:52:37,747 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown/_temporary
2016-05-17 21:52:37,748 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.411
2016-05-17 21:52:37,798 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-17 21:52:37,824 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:37,824 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:37,874 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-17 21:52:37,893 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-05-17 21:52:37,894 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:37,894 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 21:52:37,894 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:37,918 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-05-17 21:52:37,928 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 21:52:37,930 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:37,930 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:37,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-17 21:52:38,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-17 21:52:38,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:38,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:38,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-17 21:52:38,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] parallel execution of steps is enabled: false
2016-05-17 21:52:38,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executing total steps: 1
2016-05-17 21:52:38,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] allocating management threads: 1
2016-05-17 21:52:38,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38,080 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:38,099 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-17 21:52:38,105 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:38,105 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:38,105 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:38,117 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001 doesn't exist and is created
2016-05-17 21:52:38,141 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:38,141 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:38,141 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001
2016-05-17 21:52:38,148 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521958105_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:38,165 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:38,167 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:38,169 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:38,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:38,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001/localmode-log-dir
2016-05-17 21:52:38,171 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001/localmode-log-dir/history.txt.appattempt_1463521958105_0001_000000, maxErrors=10
2016-05-17 21:52:38,171 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:38,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521958105_0001_000000, appSubmitTime=1463521958148, launchTime=1463521958148
2016-05-17 21:52:38,176 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:38,178 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:38,179 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41024
2016-05-17 21:52:38,181 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:38,182 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:38,181 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:38,186 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521958105_0001_000000, startTime=1463521958186
2016-05-17 21:52:38,186 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:38,187 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:38,241 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:38,241 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:38,241 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463521958105_0001, dagName=[984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38,259 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38,261 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521958105_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001/localmode-log-dir/dag_1463521958105_0001_1.dot
2016-05-17 21:52:38,262 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001/localmode-log-dir/dag_1463521958105_0001_1-tez-dag.pb.txt
2016-05-17 21:52:38,272 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521958105_0001_1
2016-05-17 21:52:38,272 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,273 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38 Running Dag: dag_1463521958105_0001_1
2016-05-17 21:52:38,286 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:38,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521958105_0001_1, submitTime=1463521958260
2016-05-17 21:52:38,287 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,288 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:38,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:38,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521958105_0001_1, initTime=1463521958287
2016-05-17 21:52:38,289 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521958105_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:38,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_STARTED]: dagID=dag_1463521958105_0001_1, startTime=1463521958289
2016-05-17 21:52:38,289 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:38,290 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521958105_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:38,290 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CB9CB66FFDF04BAE947D0404B2C770ED : {76974C0A371B4F67B76EE342D4C1383F={InputName=76974C0A371B4F67B76EE342D4C1383F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:38,290 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 76974C0A371B4F67B76EE342D4C1383F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:38,290 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,290 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,290 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,290 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]. Starting root input initializers: 1
2016-05-17 21:52:38,291 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:38,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:38,291 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D761E2268808497B880B1FD23B861F3E : {8B20E3D0427141118675F82D414BEA9E={InputName=8B20E3D0427141118675F82D414BEA9E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:38,291 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8B20E3D0427141118675F82D414BEA9E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:38,293 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 76974C0A371B4F67B76EE342D4C1383F on vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,292 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,295 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,295 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,295 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463521958105_0001, dagName=[984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 21:52:38,295 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]. Starting root input initializers: 1
2016-05-17 21:52:38,295 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:38,295 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:38,296 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 76974C0A371B4F67B76EE342D4C1383F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:38,296 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8B20E3D0427141118675F82D414BEA9E on vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,296 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,297 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:38,297 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:38,299 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:38,299 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 76974C0A371B4F67B76EE342D4C1383F on vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,300 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:38,300 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,300 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,300 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,300 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=799AFDC84B614A6B903FCE602360466E, vertexId=vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:38,300 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=799AFDC84B614A6B903FCE602360466E, vertexId=vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,302 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:38,302 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8B20E3D0427141118675F82D414BEA9E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:38,302 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=799AFDC84B614A6B903FCE602360466E, vertexName=E9C825134B424E54AB93BFB8D5C11177, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:38,302 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:38,303 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=799AFDC84B614A6B903FCE602360466E, vertexId=vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,303 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:38,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:38,305 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 21:52:38,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, vertexId=vertex_1463521958105_0001_1_02, initRequestedTime=1463521958296, initedTime=1463521958304, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:38,305 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:38,305 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:38,305 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {76974C0A371B4F67B76EE342D4C1383F=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:38,305 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8B20E3D0427141118675F82D414BEA9E on vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,306 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] parallelism set to 1
2016-05-17 21:52:38,306 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:38,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521958105_0001_1_01, eventCount=1
2016-05-17 21:52:38,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:38,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, vertexId=vertex_1463521958105_0001_1_01, initRequestedTime=1463521958290, initedTime=1463521958307, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:38,307 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] with distanceFromRoot: 0
2016-05-17 21:52:38,307 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:38,307 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:38,307 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8B20E3D0427141118675F82D414BEA9E=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:38,308 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] parallelism set to 1
2016-05-17 21:52:38,308 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:38,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521958105_0001_1_00, eventCount=1
2016-05-17 21:52:38,308 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:38,309 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D761E2268808497B880B1FD23B861F3E, vertexId=vertex_1463521958105_0001_1_00, initRequestedTime=1463521958291, initedTime=1463521958308, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:38,309 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] with distanceFromRoot: 0
2016-05-17 21:52:38,309 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:38,309 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,309 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:38,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521958105_0001_1_01, startRequestedTime=1463521958309, startedTime=1463521958309
2016-05-17 21:52:38,310 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:38,310 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,310 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:38,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521958105_0001_1_00, startRequestedTime=1463521958310, startedTime=1463521958310
2016-05-17 21:52:38,311 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:38,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:38,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_STARTED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, taskId=task_1463521958105_0001_1_01_000000, scheduledTime=1463521958311, launchTime=1463521958311
2016-05-17 21:52:38,312 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:38,312 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521958105_0001_1_01 for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] numStartedSources: 1 numSources: 2
2016-05-17 21:52:38,312 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] numStartedSources: 1 numSources: 2
2016-05-17 21:52:38,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:38,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_STARTED]: vertexName=D761E2268808497B880B1FD23B861F3E, taskId=task_1463521958105_0001_1_00_000000, scheduledTime=1463521958312, launchTime=1463521958312
2016-05-17 21:52:38,313 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:38,313 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521958105_0001_1_00 for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] numStartedSources: 2 numSources: 2
2016-05-17 21:52:38,313 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] with distanceFromRoot: 1
2016-05-17 21:52:38,313 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,313 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,314 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CB9CB66FFDF04BAE947D0404B2C770ED in vertex: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,314 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D761E2268808497B880B1FD23B861F3E in vertex: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,314 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E9C825134B424E54AB93BFB8D5C11177 with 2 source tasks and 1 pending tasks
2016-05-17 21:52:38,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:38,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521958105_0001_1_02, startRequestedTime=1463521958314, startedTime=1463521958314
2016-05-17 21:52:38,315 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:38,315 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: CB9CB66FFDF04BAE947D0404B2C770ED, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=76974C0A371B4F67B76EE342D4C1383F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:38,315 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:38,315 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: D761E2268808497B880B1FD23B861F3E, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8B20E3D0427141118675F82D414BEA9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:38,316 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:38,316 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:38,316 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:38,316 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463521958105_0001, with dag id: dag_1463521958105_0001_1
2016-05-17 21:52:38,317 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:38,317 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:38,318 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:38,318 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521958105_0001_00_000001
2016-05-17 21:52:38,318 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:38,322 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521958105_0001_00_000001
2016-05-17 21:52:38,323 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:38,323 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521958105_0001_00_000002
2016-05-17 21:52:38,323 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:38,324 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521958105_0001_00_000002
2016-05-17 21:52:38,324 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:38,325 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:38,325 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:38,325 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:38,325 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521958105_0001_00_000002, launchTime=1463521958325
2016-05-17 21:52:38,326 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000002
2016-05-17 21:52:38,326 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:38,326 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521958105_0001_00_000001, launchTime=1463521958325
2016-05-17 21:52:38,327 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521958105_0001_1_00_000000_0] to container: [container_1463521958105_0001_00_000002]
2016-05-17 21:52:38,327 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:38,328 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521958105_0001_1_00_000000_0] started. Is using containerId: [container_1463521958105_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:38,328 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000001
2016-05-17 21:52:38,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D761E2268808497B880B1FD23B861F3E, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0, startTime=1463521958328, containerId=container_1463521958105_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521958105_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:38,329 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:38,329 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:38,329 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:38,330 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521958105_0001_00_000002 given task: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,330 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521958105_0001_1_01_000000_0] to container: [container_1463521958105_0001_00_000001]
2016-05-17 21:52:38,330 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38 Starting to run new task attempt: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,330 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:38,331 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,331 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,331 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:38,331 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:38,332 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:38,332 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:38,332 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:38,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: D761E2268808497B880B1FD23B861F3E, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8B20E3D0427141118675F82D414BEA9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:38,332 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:38,333 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521958105_0001_00_000001 given task: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,333 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,333 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521958105_0001_1_01_000000_0] started. Is using containerId: [container_1463521958105_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:38,334 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,334 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:38 Starting to run new task attempt: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,334 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,334 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,334 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0, startTime=1463521958333, containerId=container_1463521958105_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521958105_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:38,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:38,334 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:38,336 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:38,337 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:38,337 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:38,337 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:38,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: CB9CB66FFDF04BAE947D0404B2C770ED, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=76974C0A371B4F67B76EE342D4C1383F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:38,337 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:38,336 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,340 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521958105_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:38,336 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521958105_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:38,336 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:38,341 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:38,338 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:38,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:38,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8B20E3D0427141118675F82D414BEA9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:38,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:38,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E9C825134B424E54AB93BFB8D5C11177, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:38,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=76974C0A371B4F67B76EE342D4C1383F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:38,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:38,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,345 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:38,347 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:38,347 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:38,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,596 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,595 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:38,597 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,595 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:38,597 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:38,346 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:38,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:38,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,599 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,602 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:38,602 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:38,602 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,602 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,607 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D761E2268808497B880B1FD23B861F3E, ordinal: 1
2016-05-17 21:52:38,607 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D761E2268808497B880B1FD23B861F3E, mem on start (mb), free: 345, total: 370, max: 672
2016-05-17 21:52:38,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:38,608 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:38,608 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:38,608 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:38,608 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:38,609 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:38,609 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:38,609 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:38,609 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:38,610 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8B20E3D0427141118675F82D414BEA9E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:38,610 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E9C825134B424E54AB93BFB8D5C11177, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:38,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:38,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,610 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:38,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8B20E3D0427141118675F82D414BEA9E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:38,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:38,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:38,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:38,612 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CB9CB66FFDF04BAE947D0404B2C770ED, ordinal: 0
2016-05-17 21:52:38,612 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:38,613 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3251bd19
2016-05-17 21:52:38,612 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,612 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CB9CB66FFDF04BAE947D0404B2C770ED, mem on start (mb), free: 344, total: 370, max: 672
2016-05-17 21:52:38,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:38,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:38,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:38,616 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:38,616 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:38,617 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:38,617 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:38,618 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:38,618 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:38,619 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:38,617 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3251bd19
2016-05-17 21:52:38,619 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:38,619 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:38,619 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 8B20E3D0427141118675F82D414BEA9E
2016-05-17 21:52:38,619 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,620 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,619 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E9C825134B424E54AB93BFB8D5C11177, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:38,620 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 76974C0A371B4F67B76EE342D4C1383F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:38,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:38,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 76974C0A371B4F67B76EE342D4C1383F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:38,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:38,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:38,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,621 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:38,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:38,622 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,624 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:38,626 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@69ee2790
2016-05-17 21:52:38,625 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@486f9a41; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@16f8468e; comparator=cascading.tuple.hadoop.util.TupleComparator@f93887; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:38,625 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:38,627 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@69ee2790
2016-05-17 21:52:38,628 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:38,628 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:38,627 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:38,629 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:38,629 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:38,628 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: 76974C0A371B4F67B76EE342D4C1383F
2016-05-17 21:52:38,630 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,630 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,634 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@40834bca; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6be4dbdc; comparator=cascading.tuple.hadoop.util.TupleComparator@7e1e7fda; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:38,646 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:38,647 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:38,647 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:38,646 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:38,649 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:38,649 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:38,649 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:38,649 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:38,651 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 21:52:38,651 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:38,652 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D761E2268808497B880B1FD23B861F3E, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:38,662 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:38,662 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:38,662 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:38,663 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:38,663 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:38,665 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 21:52:38,665 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:38,665 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CB9CB66FFDF04BAE947D0404B2C770ED, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:38,666 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,666 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D761E2268808497B880B1FD23B861F3E, mem on close (mb), free: 141, total: 370, max: 672
2016-05-17 21:52:38,666 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:38,667 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:38,667 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:38,667 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:38,669 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,669 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CB9CB66FFDF04BAE947D0404B2C770ED, mem on close (mb), free: 141, total: 370, max: 672
2016-05-17 21:52:38,669 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:38,670 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:38,670 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 21:52:38,670 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 21:52:38,671 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:38,673 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:38 Completed running task attempt: attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,678 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:38 Completed running task attempt: attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,681 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:38,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521958666, Process_Duration=3, Process_End_Time=1463521958669, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:38,682 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:38,682 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,682 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000001
2016-05-17 21:52:38,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521958105_0001_1_00, eventCount=1
2016-05-17 21:52:38,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521958105_0001_1_01, eventCount=1
2016-05-17 21:52:38,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:38,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D761E2268808497B880B1FD23B861F3E, taskAttemptId=attempt_1463521958105_0001_1_00_000000_0, startTime=1463521958328, finishTime=1463521958683, timeTaken=355, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958652, Process_Duration=14, Process_End_Time=1463521958666, Read_Duration=11, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:38,684 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:38,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:38,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, taskAttemptId=attempt_1463521958105_0001_1_01_000000_0, startTime=1463521958333, finishTime=1463521958684, timeTaken=351, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958666, Process_Duration=3, Process_End_Time=1463521958669, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:38,685 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:38,685 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:38,683 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521958652, Process_Duration=14, Process_End_Time=1463521958666, Read_Duration=11, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 21:52:38,685 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521958105_0001_1_00_000000_0
2016-05-17 21:52:38,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:38,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_FINISHED]: vertexName=D761E2268808497B880B1FD23B861F3E, taskId=task_1463521958105_0001_1_00_000000, startTime=1463521958328, finishTime=1463521958686, timeTaken=358, status=SUCCEEDED, successfulAttemptID=attempt_1463521958105_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958652, Process_Duration=14, Process_End_Time=1463521958666, Read_Duration=11, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:38,687 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:38,687 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:38,687 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521958105_0001_1_01_000000_0
2016-05-17 21:52:38,688 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:38,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_FINISHED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, taskId=task_1463521958105_0001_1_01_000000, startTime=1463521958333, finishTime=1463521958687, timeTaken=354, status=SUCCEEDED, successfulAttemptID=attempt_1463521958105_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958666, Process_Duration=3, Process_End_Time=1463521958669, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 21:52:38,688 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:38,688 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] : 1
2016-05-17 21:52:38,688 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:38,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:38,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_FINISHED]: vertexName=D761E2268808497B880B1FD23B861F3E, vertexId=vertex_1463521958105_0001_1_00, initRequestedTime=1463521958291, initedTime=1463521958308, startRequestedTime=1463521958310, startedTime=1463521958310, finishTime=1463521958688, timeTaken=378, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958652, Process_Duration=14, Process_End_Time=1463521958666, Read_Duration=11, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521958328, firstTasksToStart=[ task_1463521958105_0001_1_00_000000 ], lastTaskFinishTime=1463521958683, lastTasksToFinish=[ task_1463521958105_0001_1_00_000000 ], minTaskDuration=355, maxTaskDuration=355, avgTaskDuration=355.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521958105_0001_1_00_000000 ], longestDurationTasks=[ task_1463521958105_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:38,690 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:38,690 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:38,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521958105_0001_00_000002, stoppedTime=1463521958690, exitStatus=0
2016-05-17 21:52:38,690 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:38,690 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:38,691 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] : 1
2016-05-17 21:52:38,691 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:38,691 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:38,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_FINISHED]: vertexName=CB9CB66FFDF04BAE947D0404B2C770ED, vertexId=vertex_1463521958105_0001_1_01, initRequestedTime=1463521958290, initedTime=1463521958307, startRequestedTime=1463521958309, startedTime=1463521958309, finishTime=1463521958691, timeTaken=382, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=774, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521958666, Process_Duration=3, Process_End_Time=1463521958669, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463521958333, firstTasksToStart=[ task_1463521958105_0001_1_01_000000 ], lastTaskFinishTime=1463521958684, lastTasksToFinish=[ task_1463521958105_0001_1_01_000000 ], minTaskDuration=351, maxTaskDuration=351, avgTaskDuration=351.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521958105_0001_1_01_000000 ], longestDurationTasks=[ task_1463521958105_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:38,692 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:38,692 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:38,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521958105_0001_00_000001, stoppedTime=1463521958692, exitStatus=0
2016-05-17 21:52:38,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:38,692 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:38,693 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,692 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:38,693 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000002
2016-05-17 21:52:38,693 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] attempt: attempt_1463521958105_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:38,693 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521958105_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:38,693 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:38,693 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521958105_0001_00_000002, Exiting
2016-05-17 21:52:38,693 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521958105_0001_00_000002
2016-05-17 21:52:38,694 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521958105_0001_00_000002
2016-05-17 21:52:38,693 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:38,699 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521958105_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:38,700 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000001: 18 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:38,700 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521958105_0001_00_000001, Exiting
2016-05-17 21:52:38,700 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521958105_0001_00_000001
2016-05-17 21:52:38,700 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521958105_0001_00_000001
2016-05-17 21:52:38,700 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521958105_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:38,700 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521958105_0001_00_000002 completed successfully
2016-05-17 21:52:38,701 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521958105_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:38,701 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521958105_0001_00_000001 completed successfully
2016-05-17 21:52:38,701 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521958105_0001_00_000002
2016-05-17 21:52:38,702 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] attempt: attempt_1463521958105_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:38,702 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E9C825134B424E54AB93BFB8D5C11177
2016-05-17 21:52:38,702 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,702 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:38,703 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521958105_0001_00_000001
2016-05-17 21:52:38,703 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:38,703 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:38,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_STARTED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, taskId=task_1463521958105_0001_1_02_000000, scheduledTime=1463521958704, launchTime=1463521958704
2016-05-17 21:52:38,704 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:38,705 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:38,705 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: E9C825134B424E54AB93BFB8D5C11177, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D761E2268808497B880B1FD23B861F3E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=CB9CB66FFDF04BAE947D0404B2C770ED, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=799AFDC84B614A6B903FCE602360466E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:38,705 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:38,705 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:38,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:38,706 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521958105_0001_00_000003
2016-05-17 21:52:38,706 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:38,708 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521958105_0001_00_000003
2016-05-17 21:52:38,708 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:38,710 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:38,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521958105_0001_00_000003, launchTime=1463521958710
2016-05-17 21:52:38,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:38,711 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000003
2016-05-17 21:52:38,712 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521958105_0001_1_02_000000_0] to container: [container_1463521958105_0001_00_000003]
2016-05-17 21:52:38,713 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:38,713 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521958105_0001_00_000003 given task: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,714 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38 Starting to run new task attempt: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,713 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521958105_0001_1_02_000000_0] started. Is using containerId: [container_1463521958105_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 21:52:38,714 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0, startTime=1463521958713, containerId=container_1463521958105_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521958105_0001_00_000003/teamcity, completedLogs=
2016-05-17 21:52:38,715 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:38,715 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:38,716 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:38,716 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:38,717 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:38,717 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:38,717 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:38,717 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:38,718 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:38,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: E9C825134B424E54AB93BFB8D5C11177, VertexParallelism: 1, TaskAttemptID:attempt_1463521958105_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D761E2268808497B880B1FD23B861F3E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=CB9CB66FFDF04BAE947D0404B2C770ED, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=799AFDC84B614A6B903FCE602360466E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:38,718 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:38,721 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:38,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,723 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521958105_0001_1_02_000000_0 sent events: (0-2)
2016-05-17 21:52:38,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CB9CB66FFDF04BAE947D0404B2C770ED, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:38,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:38,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=799AFDC84B614A6B903FCE602360466E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:38,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:38,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 799AFDC84B614A6B903FCE602360466E
2016-05-17 21:52:38,726 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:38,727 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:38,727 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:38,727 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,727 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 799AFDC84B614A6B903FCE602360466E
2016-05-17 21:52:38,727 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:38,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D761E2268808497B880B1FD23B861F3E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:38,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:38,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,734 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:38,735 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:38,735 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,740 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E9C825134B424E54AB93BFB8D5C11177, ordinal: 2
2016-05-17 21:52:38,740 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9C825134B424E54AB93BFB8D5C11177, mem on start (mb), free: 133, total: 370, max: 672
2016-05-17 21:52:38,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:38,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-05-17 21:52:38,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:38,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:38,743 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:38,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 799AFDC84B614A6B903FCE602360466E
2016-05-17 21:52:38,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:38,743 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:38,744 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:38,744 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:38,744 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:38,744 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:38,745 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-17 21:52:38,745 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-05-17 21:52:38,745 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CB9CB66FFDF04BAE947D0404B2C770ED, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 21:52:38,746 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 799AFDC84B614A6B903FCE602360466E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:38,746 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D761E2268808497B880B1FD23B861F3E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 21:52:38,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:38,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D761E2268808497B880B1FD23B861F3E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:38,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CB9CB66FFDF04BAE947D0404B2C770ED being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:38,748 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:38,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-05-17 21:52:38,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-05-17 21:52:38,748 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:38,749 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:38,749 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:38,749 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D761E2268808497B880B1FD23B861F3E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:38,749 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 21:52:38,749 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CB9CB66FFDF04BAE947D0404B2C770ED with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:38,750 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:38,750 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 21:52:38,750 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:38,750 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:38,751 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:38,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:38,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:38,753 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:38,753 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521958105_0001_1_00_000000_0_10055, runDuration: 325595, ]
2016-05-17 21:52:38,753 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521958105_0001_1_01_000000_0_10057, runDuration: 331364, ]
2016-05-17 21:52:38,756 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521958105_0001_1_01_000000_0_10057] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521958756, TimeTaken=2, Rate=0.05 MB/s
2016-05-17 21:52:38,756 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,757 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2016-05-17 21:52:38,757 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:38,757 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CB9CB66FFDF04BAE947D0404B2C770ED] #9 in 3ms
2016-05-17 21:52:38,757 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] streamed: true, id: 0D56736F1639452F883D131E65A15ADC
2016-05-17 21:52:38,759 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CB9CB66FFDF04BAE947D0404B2C770ED] #9, status:false, isInterrupted:false
2016-05-17 21:52:38,759 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"], id: 799AFDC84B614A6B903FCE602360466E
2016-05-17 21:52:38,760 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:38,760 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-17 21:52:38,760 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 0D56736F1639452F883D131E65A15ADC, for 2 inputs
2016-05-17 21:52:38,760 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 0D56736F1639452F883D131E65A15ADC, for 2 inputs
2016-05-17 21:52:38,761 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:38,761 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:38,761 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:38,762 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:38,762 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:38,762 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:38,760 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521958105_0001_1_00_000000_0_10055] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463521958760, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 21:52:38,765 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:38,765 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:38,764 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 21:52:38,766 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 21:52:38,766 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D761E2268808497B880B1FD23B861F3E] #10 in 7ms
2016-05-17 21:52:38,766 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D761E2268808497B880B1FD23B861F3E] #10, status:false, isInterrupted:false
2016-05-17 21:52:39,046 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:39,046 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:39,047 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:39,048 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 21:52:39,048 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:39,049 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:39,049 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 21:52:39,049 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:39,049 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:39,049 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E9C825134B424E54AB93BFB8D5C11177, all 2 inputs ready in: 00:00:00.283
2016-05-17 21:52:39,051 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@30bad9de; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@3b3596d0
2016-05-17 21:52:39,054 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@3b40500e; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7b47e585
2016-05-17 21:52:39,057 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:39,057 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:39,058 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39,058 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521958105_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 21:52:39,059 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39,060 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9C825134B424E54AB93BFB8D5C11177, mem on close (mb), free: 519, total: 543, max: 672
2016-05-17 21:52:39,060 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D761E2268808497B880B1FD23B861F3E
2016-05-17 21:52:39,060 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:39,062 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:39,062 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:39,062 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:39,062 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:39,062 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CB9CB66FFDF04BAE947D0404B2C770ED
2016-05-17 21:52:39,063 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:39,064 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:39,064 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:39,064 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:39,064 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:39 Completed running task attempt: attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39,065 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:39,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=280, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521959050, Process_Duration=9, Process_End_Time=1463521959059, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 21:52:39,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, taskAttemptId=attempt_1463521958105_0001_1_02_000000_0, startTime=1463521958713, finishTime=1463521959066, timeTaken=353, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=280, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521959050, Process_Duration=9, Process_End_Time=1463521959059, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:39,067 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:39,067 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:39,068 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,068 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521958105_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:39,069 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521958105_0001_00_000003
2016-05-17 21:52:39,069 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521958105_0001_1_02_000000_0
2016-05-17 21:52:39,069 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:39,069 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:TASK_FINISHED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, taskId=task_1463521958105_0001_1_02_000000, startTime=1463521958713, finishTime=1463521959069, timeTaken=356, status=SUCCEEDED, successfulAttemptID=attempt_1463521958105_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=280, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521959050, Process_Duration=9, Process_End_Time=1463521959059, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 21:52:39,069 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521958105_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:39,070 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] : 1
2016-05-17 21:52:39,070 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:39,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:VERTEX_FINISHED]: vertexName=E9C825134B424E54AB93BFB8D5C11177, vertexId=vertex_1463521958105_0001_1_02, initRequestedTime=1463521958296, initedTime=1463521958304, startRequestedTime=1463521958314, startedTime=1463521958314, finishTime=1463521959070, timeTaken=756, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=280, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521959050, Process_Duration=9, Process_End_Time=1463521959059, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463521958713, firstTasksToStart=[ task_1463521958105_0001_1_02_000000 ], lastTaskFinishTime=1463521959066, lastTasksToFinish=[ task_1463521958105_0001_1_02_000000 ], minTaskDuration=353, maxTaskDuration=353, avgTaskDuration=353.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521958105_0001_1_02_000000 ], longestDurationTasks=[ task_1463521958105_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:39,071 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:39,071 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:39,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521958105_0001_00_000003, stoppedTime=1463521959071, exitStatus=0
2016-05-17 21:52:39,071 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:39,071 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:39,071 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 21:52:39,072 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521958105_0001_1
2016-05-17 21:52:39,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521958105_0001_1
2016-05-17 21:52:39,072 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 799AFDC84B614A6B903FCE602360466E for vertex: vertex_1463521958105_0001_1_02 [E9C825134B424E54AB93BFB8D5C11177]
2016-05-17 21:52:39,072 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:39,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:39,079 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521958105_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 21:52:39,079 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521958105_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:39,080 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521958105_0001_00_000003, Exiting
2016-05-17 21:52:39,080 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521958105_0001_00_000003
2016-05-17 21:52:39,080 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521958105_0001_00_000003
2016-05-17 21:52:39,080 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521958105_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:39,081 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521958105_0001_00_000003 completed successfully
2016-05-17 21:52:39,081 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521958105_0001_1_01 [CB9CB66FFDF04BAE947D0404B2C770ED]
2016-05-17 21:52:39,081 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521958105_0001_1_00 [D761E2268808497B880B1FD23B861F3E]
2016-05-17 21:52:39,084 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521958105_0001_1, queueSize=0
2016-05-17 21:52:39,084 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:39,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521958105_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521958105_0001_1, startTime=1463521958289, finishTime=1463521959081, timeTaken=792, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=470, FILE_BYTES_WRITTEN=1600, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=770, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1346895872, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=300, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390565876368, Process_Duration=26, Process_End_Time=4390565876394, Read_Duration=13, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-17 21:52:39,084 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521958105_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:39,084 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521958105_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:39,085 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521958105_0001_00_000003
2016-05-17 21:52:39,085 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521958105_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:39 Completed Dag: dag_1463521958105_0001_1
2016-05-17 21:52:39,086 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521958105_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:39,086 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521958105_0001_1_post
2016-05-17 21:52:39,086 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,086 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:39,093 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463521958105_0001
2016-05-17 21:52:39,096 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:39,096 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:39,096 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:39,096 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:39,096 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:39,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:39,097 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:39,097 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:39,097 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:39,097 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:39,098 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:39,098 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:39,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:39,104 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/984DEB678D844BDAAA1EB2F92D0FA582/CFCE69B8E71B418AA0A92366377D27B0/.tez/application_1463521958105_0001
2016-05-17 21:52:39,104 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup/_temporary
2016-05-17 21:52:39,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] completed in: 00:01.028
2016-05-17 21:52:39,190 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-17 21:52:39,213 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:39,213 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:39,250 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:52:39,259 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 21:52:39,259 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:39,259 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:39,260 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:39,276 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:39,277 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-17 21:52:39,285 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:39,285 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:39,285 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-17 21:52:39,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:39,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:39,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:39,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...tform/testReplace/replace
2016-05-17 21:52:39,287 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:39,298 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:39,302 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:39,303 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:39,303 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:39,312 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001 doesn't exist and is created
2016-05-17 21:52:39,328 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:39,328 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:39,329 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001
2016-05-17 21:52:39,332 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521959303_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:39,343 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:39,344 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:39,345 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:39,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:39,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001/localmode-log-dir
2016-05-17 21:52:39,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001/localmode-log-dir/history.txt.appattempt_1463521959303_0001_000000, maxErrors=10
2016-05-17 21:52:39,346 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:39,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521959303_0001_000000, appSubmitTime=1463521959332, launchTime=1463521959332
2016-05-17 21:52:39,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:39,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:39,352 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:39,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:39,357 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:32975
2016-05-17 21:52:39,358 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:39,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521959303_0001_000000, startTime=1463521959360
2016-05-17 21:52:39,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:39,360 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:39,429 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:39,429 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:39,429 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463521959303_0001, dagName=[65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace
2016-05-17 21:52:39,452 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace
2016-05-17 21:52:39,454 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521959303_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001/localmode-log-dir/dag_1463521959303_0001_1.dot
2016-05-17 21:52:39,459 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001/localmode-log-dir/dag_1463521959303_0001_1-tez-dag.pb.txt
2016-05-17 21:52:39,465 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521959303_0001_1
2016-05-17 21:52:39,465 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,465 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace
2016-05-17 21:52:39 Running Dag: dag_1463521959303_0001_1
2016-05-17 21:52:39,477 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521959303_0001_1, submitTime=1463521959453
2016-05-17 21:52:39,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:39,478 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C8389DC629D94E66B1956D1FFCAFA945
2016-05-17 21:52:39,478 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:39,479 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521959303_0001_1, initTime=1463521959477
2016-05-17 21:52:39,479 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:39,479 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959303_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:39,479 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_STARTED]: dagID=dag_1463521959303_0001_1, startTime=1463521959479
2016-05-17 21:52:39,480 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:39,480 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:39,480 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959303_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:39,481 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C8389DC629D94E66B1956D1FFCAFA945 : {85AA824A7BA14D46A42C4E1E5B430590={InputName=85AA824A7BA14D46A42C4E1E5B430590}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:39,481 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 85AA824A7BA14D46A42C4E1E5B430590, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:39,481 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,481 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463521959303_0001, dagName=[65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace
2016-05-17 21:52:39,481 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,482 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,483 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]. Starting root input initializers: 1
2016-05-17 21:52:39,485 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:39,485 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 85AA824A7BA14D46A42C4E1E5B430590 on vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,487 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:39,488 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 85AA824A7BA14D46A42C4E1E5B430590 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:39,488 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:39,488 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:39,491 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:39,491 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 85AA824A7BA14D46A42C4E1E5B430590 on vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,492 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:39,492 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {85AA824A7BA14D46A42C4E1E5B430590=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:39,492 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] parallelism set to 1
2016-05-17 21:52:39,492 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:39,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521959303_0001_1_00, eventCount=1
2016-05-17 21:52:39,493 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,493 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2B86812B7FBC4CD59E4441D15694232E, vertexId=vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:39,493 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2B86812B7FBC4CD59E4441D15694232E, vertexId=vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,495 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2B86812B7FBC4CD59E4441D15694232E, vertexName=C8389DC629D94E66B1956D1FFCAFA945, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:39,495 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2B86812B7FBC4CD59E4441D15694232E, vertexId=vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:39,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, vertexId=vertex_1463521959303_0001_1_00, initRequestedTime=1463521959481, initedTime=1463521959496, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:39,497 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] with distanceFromRoot: 0
2016-05-17 21:52:39,497 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:39,497 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C8389DC629D94E66B1956D1FFCAFA945
2016-05-17 21:52:39,497 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:39,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521959303_0001_1_00, startRequestedTime=1463521959497, startedTime=1463521959497
2016-05-17 21:52:39,498 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:39,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:39,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:TASK_STARTED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, taskId=task_1463521959303_0001_1_00_000000, scheduledTime=1463521959498, launchTime=1463521959498
2016-05-17 21:52:39,503 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959303_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:39,503 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,505 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace, VertexName: C8389DC629D94E66B1956D1FFCAFA945, VertexParallelism: 1, TaskAttemptID:attempt_1463521959303_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=85AA824A7BA14D46A42C4E1E5B430590, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2B86812B7FBC4CD59E4441D15694232E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:39,505 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959303_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:39,505 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:39,506 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:39,506 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:39,507 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:39,507 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521959303_0001_00_000001
2016-05-17 21:52:39,507 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:39,509 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521959303_0001, with dag id: dag_1463521959303_0001_1
2016-05-17 21:52:39,509 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521959303_0001_00_000001
2016-05-17 21:52:39,510 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:39,510 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:39,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:39,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521959303_0001_00_000001, launchTime=1463521959510
2016-05-17 21:52:39,512 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959303_0001_00_000001
2016-05-17 21:52:39,514 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521959303_0001_1_00_000000_0] to container: [container_1463521959303_0001_00_000001]
2016-05-17 21:52:39,514 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:39,514 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521959303_0001_1_00_000000_0] started. Is using containerId: [container_1463521959303_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:39,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0, startTime=1463521959514, containerId=container_1463521959303_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521959303_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:39,515 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959303_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:39,515 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959303_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:39,515 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:39,517 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521959303_0001_00_000001 given task: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,517 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959303_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39 Starting to run new task attempt: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,517 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,518 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,518 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:39,518 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:39,518 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:39,518 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:39,519 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:39,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6] test/(1/1) ...tform/testReplace/replace, VertexName: C8389DC629D94E66B1956D1FFCAFA945, VertexParallelism: 1, TaskAttemptID:attempt_1463521959303_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=85AA824A7BA14D46A42C4E1E5B430590, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2B86812B7FBC4CD59E4441D15694232E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:39,519 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:39,520 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:39,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2B86812B7FBC4CD59E4441D15694232E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:39,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:39,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2B86812B7FBC4CD59E4441D15694232E
2016-05-17 21:52:39,523 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:39,523 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2B86812B7FBC4CD59E4441D15694232E
2016-05-17 21:52:39,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=85AA824A7BA14D46A42C4E1E5B430590, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:39,523 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521959303_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:39,524 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:39,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:39,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,526 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,531 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:39,532 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,534 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:39,534 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2B86812B7FBC4CD59E4441D15694232E
2016-05-17 21:52:39,535 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8389DC629D94E66B1956D1FFCAFA945, ordinal: 0
2016-05-17 21:52:39,536 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8389DC629D94E66B1956D1FFCAFA945, mem on start (mb), free: 442, total: 543, max: 672
2016-05-17 21:52:39,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:39,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:39,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:39,536 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:39,536 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:39,538 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:39,538 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:39,538 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:39,538 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:39,539 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:39,539 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2B86812B7FBC4CD59E4441D15694232E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:39,539 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 85AA824A7BA14D46A42C4E1E5B430590, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:39,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:39,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,539 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:39,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 85AA824A7BA14D46A42C4E1E5B430590 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:39,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:39,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:39,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:39,541 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,542 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:39,542 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@65bfce1d
2016-05-17 21:52:39,543 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@65bfce1d
2016-05-17 21:52:39,543 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:39,543 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:39,544 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:39,544 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 85AA824A7BA14D46A42C4E1E5B430590
2016-05-17 21:52:39,545 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"], id: 2B86812B7FBC4CD59E4441D15694232E
2016-05-17 21:52:39,545 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-17 21:52:39,545 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:39,545 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:39,545 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8389DC629D94E66B1956D1FFCAFA945, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:39,547 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:39,547 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:39,548 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,548 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521959303_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:39,549 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,549 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8389DC629D94E66B1956D1FFCAFA945, mem on close (mb), free: 440, total: 543, max: 672
2016-05-17 21:52:39 Completed running task attempt: attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,549 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:39,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-17 21:52:39,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:39,550 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:39,550 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,551 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959303_0001_00_000001
2016-05-17 21:52:39,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, taskAttemptId=attempt_1463521959303_0001_1_00_000000_0, startTime=1463521959514, finishTime=1463521959550, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:39,551 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959303_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:39,551 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:39,551 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521959303_0001_1_00_000000_0
2016-05-17 21:52:39,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:39,553 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:TASK_FINISHED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, taskId=task_1463521959303_0001_1_00_000000, startTime=1463521959514, finishTime=1463521959553, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463521959303_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:39,554 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959303_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:39,554 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] : 1
2016-05-17 21:52:39,554 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:VERTEX_FINISHED]: vertexName=C8389DC629D94E66B1956D1FFCAFA945, vertexId=vertex_1463521959303_0001_1_00, initRequestedTime=1463521959481, initedTime=1463521959496, startRequestedTime=1463521959497, startedTime=1463521959497, finishTime=1463521959554, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463521959514, firstTasksToStart=[ task_1463521959303_0001_1_00_000000 ], lastTaskFinishTime=1463521959550, lastTasksToFinish=[ task_1463521959303_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521959303_0001_1_00_000000 ], longestDurationTasks=[ task_1463521959303_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:39,556 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:39,556 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:39,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521959303_0001_00_000001, stoppedTime=1463521959556, exitStatus=0
2016-05-17 21:52:39,556 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:39,556 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:39,556 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521959303_0001_1
2016-05-17 21:52:39,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521959303_0001_1
2016-05-17 21:52:39,557 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2B86812B7FBC4CD59E4441D15694232E for vertex: vertex_1463521959303_0001_1_00 [C8389DC629D94E66B1956D1FFCAFA945]
2016-05-17 21:52:39,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:39,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:39,561 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521959303_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:39,561 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959303_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:39,561 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521959303_0001_00_000001, Exiting
2016-05-17 21:52:39,561 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521959303_0001_00_000001
2016-05-17 21:52:39,562 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521959303_0001_00_000001
2016-05-17 21:52:39,563 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521959303_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:39,564 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521959303_0001_00_000001 completed successfully
2016-05-17 21:52:39,568 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521959303_0001_1, queueSize=0
2016-05-17 21:52:39,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:39,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959303_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521959303_0001_1, startTime=1463521959479, finishTime=1463521959567, timeTaken=88, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463521959546, Process_Duration=2, Process_End_Time=1463521959548, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 21:52:39,569 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521959303_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:39,570 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959303_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:39,570 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959303_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:39,570 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521959303_0001_00_000001
2016-05-17 21:52:39 Completed Dag: dag_1463521959303_0001_1
2016-05-17 21:52:39,571 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521959303_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:39,571 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521959303_0001_1_post
2016-05-17 21:52:39,571 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,571 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:39,577 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463521959303_0001
2016-05-17 21:52:39,579 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:39,579 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:39,579 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:39,580 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:39,580 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:39,580 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:39,581 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:39,581 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:39,581 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:39,582 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:39,582 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:39,582 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:39,582 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:39,586 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/65FCC9EF511F47F1893C7DE46AEEE29F/E9201F2F0CDD4FD1AD9F5D253AB7D5D6/.tez/application_1463521959303_0001
2016-05-17 21:52:39,586 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace/_temporary
2016-05-17 21:52:39,587 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.301
2016-05-17 21:52:39,621 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-17 21:52:39,643 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:39,643 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:39,678 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:52:39,689 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 21:52:39,689 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:39,689 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:39,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:39,704 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 21:52:39,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-17 21:52:39,756 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 21:52:39,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:39,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-17 21:52:39,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-17 21:52:39,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-17 21:52:39,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-17 21:52:39,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount2/count2
2016-05-17 21:52:39,758 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:39,770 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 21:52:39,774 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:39,775 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:39,775 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:39,785 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001 doesn't exist and is created
2016-05-17 21:52:39,803 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:39,804 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:39,804 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001
2016-05-17 21:52:39,807 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521959775_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:39,817 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:39,817 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:39,819 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:39,819 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:39,819 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001/localmode-log-dir
2016-05-17 21:52:39,820 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001/localmode-log-dir/history.txt.appattempt_1463521959775_0001_000000, maxErrors=10
2016-05-17 21:52:39,820 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:39,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521959775_0001_000000, appSubmitTime=1463521959806, launchTime=1463521959807
2016-05-17 21:52:39,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:39,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:39,827 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:39,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:39,831 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54977
2016-05-17 21:52:39,833 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:39,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521959775_0001_000000, startTime=1463521959835
2016-05-17 21:52:39,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:39,835 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:39,904 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:39,904 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:39,904 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463521959775_0001, dagName=[D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2
2016-05-17 21:52:39,920 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2
2016-05-17 21:52:39,921 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521959775_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001/localmode-log-dir/dag_1463521959775_0001_1.dot
2016-05-17 21:52:39,922 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001/localmode-log-dir/dag_1463521959775_0001_1-tez-dag.pb.txt
2016-05-17 21:52:39,932 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521959775_0001_1
2016-05-17 21:52:39,932 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,932 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2
2016-05-17 21:52:39 Running Dag: dag_1463521959775_0001_1
2016-05-17 21:52:39,944 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:39,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521959775_0001_1, submitTime=1463521959920
2016-05-17 21:52:39,946 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,946 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:39,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:39,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521959775_0001_1, initTime=1463521959945
2016-05-17 21:52:39,948 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959775_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:39,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:39,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_STARTED]: dagID=dag_1463521959775_0001_1, startTime=1463521959948
2016-05-17 21:52:39,948 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:39,949 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463521959775_0001, dagName=[D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2
2016-05-17 21:52:39,949 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959775_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:39,949 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E50C8771203D4054B9211097074879AF : {69B3D97299504777947459B764F6AA36={InputName=69B3D97299504777947459B764F6AA36}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:39,949 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 69B3D97299504777947459B764F6AA36, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:39,949 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,949 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,949 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,950 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]. Starting root input initializers: 1
2016-05-17 21:52:39,950 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:39,950 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,953 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:39,953 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,953 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,953 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,953 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0FE2DE5D6F974A9CBA974E77810B04FD, vertexId=vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:39,953 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0FE2DE5D6F974A9CBA974E77810B04FD, vertexId=vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,954 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 69B3D97299504777947459B764F6AA36 on vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,955 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0FE2DE5D6F974A9CBA974E77810B04FD, vertexName=DB8E2F2045794DDEAF203EE1705A8C16, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:39,955 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0FE2DE5D6F974A9CBA974E77810B04FD, vertexId=vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:39,957 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:39,957 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:39,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, vertexId=vertex_1463521959775_0001_1_01, initRequestedTime=1463521959950, initedTime=1463521959957, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:39,957 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:39,957 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 69B3D97299504777947459B764F6AA36 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:39,958 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:39,959 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:39,960 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 21:52:39,960 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 69B3D97299504777947459B764F6AA36 on vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,960 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:39,961 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {69B3D97299504777947459B764F6AA36=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:39,961 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] parallelism set to 1
2016-05-17 21:52:39,961 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:39,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521959775_0001_1_00, eventCount=1
2016-05-17 21:52:39,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:39,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E50C8771203D4054B9211097074879AF, vertexId=vertex_1463521959775_0001_1_00, initRequestedTime=1463521959949, initedTime=1463521959962, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:39,962 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] with distanceFromRoot: 0
2016-05-17 21:52:39,963 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:39,963 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E50C8771203D4054B9211097074879AF
2016-05-17 21:52:39,963 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:39,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521959775_0001_1_00, startRequestedTime=1463521959963, startedTime=1463521959963
2016-05-17 21:52:39,964 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:39,964 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:39,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_STARTED]: vertexName=E50C8771203D4054B9211097074879AF, taskId=task_1463521959775_0001_1_00_000000, scheduledTime=1463521959964, launchTime=1463521959964
2016-05-17 21:52:39,965 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:39,965 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521959775_0001_1_00 for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] numStartedSources: 1 numSources: 1
2016-05-17 21:52:39,965 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] with distanceFromRoot: 1
2016-05-17 21:52:39,965 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:39,966 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E50C8771203D4054B9211097074879AF in vertex: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,966 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DB8E2F2045794DDEAF203EE1705A8C16 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:39,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:39,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521959775_0001_1_01, startRequestedTime=1463521959966, startedTime=1463521959966
2016-05-17 21:52:39,967 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:39,967 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2, VertexName: E50C8771203D4054B9211097074879AF, VertexParallelism: 1, TaskAttemptID:attempt_1463521959775_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=69B3D97299504777947459B764F6AA36, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DB8E2F2045794DDEAF203EE1705A8C16, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:39,967 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463521959775_0001, with dag id: dag_1463521959775_0001_1
2016-05-17 21:52:39,967 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:39,968 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:39,969 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:39,969 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:39,970 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:39,971 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521959775_0001_00_000001
2016-05-17 21:52:39,971 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:39,972 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521959775_0001_00_000001
2016-05-17 21:52:39,972 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:39,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:39,974 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:39,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521959775_0001_00_000001, launchTime=1463521959973
2016-05-17 21:52:39,975 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959775_0001_00_000001
2016-05-17 21:52:39,976 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521959775_0001_1_00_000000_0] to container: [container_1463521959775_0001_00_000001]
2016-05-17 21:52:39,976 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:39,977 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521959775_0001_00_000001 given task: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39,977 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959775_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39 Starting to run new task attempt: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39,978 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39,978 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:39,978 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:39,978 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521959775_0001_1_00_000000_0] started. Is using containerId: [container_1463521959775_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:39,978 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:39,979 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:39,979 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:39,979 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:39,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2, VertexName: E50C8771203D4054B9211097074879AF, VertexParallelism: 1, TaskAttemptID:attempt_1463521959775_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=69B3D97299504777947459B764F6AA36, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DB8E2F2045794DDEAF203EE1705A8C16, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:39,979 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:39,980 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:39,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E50C8771203D4054B9211097074879AF, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0, startTime=1463521959978, containerId=container_1463521959775_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521959775_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:39,980 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:39,980 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:39,983 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521959775_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:39,983 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:39,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:39,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,985 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=69B3D97299504777947459B764F6AA36, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:39,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:39,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:39,989 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:39,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DB8E2F2045794DDEAF203EE1705A8C16, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:39,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:39,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,992 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:39,992 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:39,992 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:39,997 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:39,997 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:39,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:39,998 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E50C8771203D4054B9211097074879AF, ordinal: 0
2016-05-17 21:52:39,998 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E50C8771203D4054B9211097074879AF, mem on start (mb), free: 358, total: 543, max: 672
2016-05-17 21:52:39,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:39,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:39,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:39,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:39,999 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:39,999 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:39,999 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:39,999 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:40,000 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:40,000 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:40,000 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 69B3D97299504777947459B764F6AA36, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:40,001 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DB8E2F2045794DDEAF203EE1705A8C16, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:40,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:40,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 69B3D97299504777947459B764F6AA36 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:40,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:40,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:40,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:40,002 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:40,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:40,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:40,003 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40,003 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:40,004 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@e59e47d
2016-05-17 21:52:40,005 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@e59e47d
2016-05-17 21:52:40,005 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:40,005 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:40,006 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,007 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 69B3D97299504777947459B764F6AA36
2016-05-17 21:52:40,007 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: E98B27387D99441FA4FC8D844CEF0506
2016-05-17 21:52:40,007 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] E98B27387D99441FA4FC8D844CEF0506
2016-05-17 21:52:40,009 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2ce0692f; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@4119a3ad; comparator=cascading.tuple.hadoop.util.TupleComparator@7a48d43e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:40,009 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:40,009 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:40,010 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:40,029 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:40,030 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:40,030 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:40,030 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:40,031 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:40,031 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,031 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:40,032 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E50C8771203D4054B9211097074879AF, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:40,034 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40,034 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E50C8771203D4054B9211097074879AF, mem on close (mb), free: 425, total: 556, max: 672
2016-05-17 21:52:40,034 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:40,035 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:40,035 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 21:52:40,035 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:52:40,039 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40,043 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:40,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521959775_0001_1_00, eventCount=1
2016-05-17 21:52:40,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:40,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E50C8771203D4054B9211097074879AF, taskAttemptId=attempt_1463521959775_0001_1_00_000000_0, startTime=1463521959978, finishTime=1463521960044, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960032, Process_Duration=2, Process_End_Time=1463521960034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:40,045 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:40,045 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521959775_0001_1_00_000000_0
2016-05-17 21:52:40,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_FINISHED]: vertexName=E50C8771203D4054B9211097074879AF, taskId=task_1463521959775_0001_1_00_000000, startTime=1463521959978, finishTime=1463521960045, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1463521959775_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960032, Process_Duration=2, Process_End_Time=1463521960034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:40,046 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:40,046 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] : 1
2016-05-17 21:52:40,046 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:40,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:40,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:40,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_FINISHED]: vertexName=E50C8771203D4054B9211097074879AF, vertexId=vertex_1463521959775_0001_1_00, initRequestedTime=1463521959949, initedTime=1463521959962, startRequestedTime=1463521959963, startedTime=1463521959963, finishTime=1463521960047, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960032, Process_Duration=2, Process_End_Time=1463521960034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463521959978, firstTasksToStart=[ task_1463521959775_0001_1_00_000000 ], lastTaskFinishTime=1463521960044, lastTasksToFinish=[ task_1463521959775_0001_1_00_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521959775_0001_1_00_000000 ], longestDurationTasks=[ task_1463521959775_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:40,048 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:40,049 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] attempt: attempt_1463521959775_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:40,049 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DB8E2F2045794DDEAF203EE1705A8C16
2016-05-17 21:52:40,049 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:40,050 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:40,049 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:40,051 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_STARTED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, taskId=task_1463521959775_0001_1_01_000000, scheduledTime=1463521960050, launchTime=1463521960050
2016-05-17 21:52:40,051 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:40,051 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:40,051 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:40,052 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:40,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521959775_0001_00_000001, stoppedTime=1463521960052, exitStatus=0
2016-05-17 21:52:40,052 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:40,052 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:40,053 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2, VertexName: DB8E2F2045794DDEAF203EE1705A8C16, VertexParallelism: 1, TaskAttemptID:attempt_1463521959775_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E50C8771203D4054B9211097074879AF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0FE2DE5D6F974A9CBA974E77810B04FD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:40,053 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:40,053 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:40,053 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521959775_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:40,054 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:40,054 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521959775_0001_00_000002
2016-05-17 21:52:40,054 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:40,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521960032, Process_Duration=2, Process_End_Time=1463521960034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:52:40,055 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:40,055 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,055 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959775_0001_00_000001
2016-05-17 21:52:40,056 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521959775_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:40,056 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959775_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:40,056 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521959775_0001_00_000001, Exiting
2016-05-17 21:52:40,056 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521959775_0001_00_000001
2016-05-17 21:52:40,057 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521959775_0001_00_000001
2016-05-17 21:52:40,056 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521959775_0001_00_000002
2016-05-17 21:52:40,060 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:40,060 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521959775_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:40,061 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521959775_0001_00_000001 completed successfully
2016-05-17 21:52:40,061 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:40,065 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:40,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:40,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521959775_0001_00_000002, launchTime=1463521960065
2016-05-17 21:52:40,066 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959775_0001_00_000002
2016-05-17 21:52:40,066 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521959775_0001_1_01_000000_0] to container: [container_1463521959775_0001_00_000002]
2016-05-17 21:52:40,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:40,067 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521959775_0001_1_01_000000_0] started. Is using containerId: [container_1463521959775_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:40,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0, startTime=1463521960067, containerId=container_1463521959775_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521959775_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:40,067 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:40,067 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:40,067 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:40,067 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521959775_0001_00_000002 given task: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,068 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959775_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40 Starting to run new task attempt: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,068 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,068 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,068 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:40,068 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:40,069 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:40,069 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:40,069 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:40,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360] count/(1/1) ...latform/testCount2/count2, VertexName: DB8E2F2045794DDEAF203EE1705A8C16, VertexParallelism: 1, TaskAttemptID:attempt_1463521959775_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E50C8771203D4054B9211097074879AF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0FE2DE5D6F974A9CBA974E77810B04FD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:40,069 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:40,070 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:40,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E50C8771203D4054B9211097074879AF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:40,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:40,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,072 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:40,073 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:40,073 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,074 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521959775_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:40,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0FE2DE5D6F974A9CBA974E77810B04FD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:40,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:40,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0FE2DE5D6F974A9CBA974E77810B04FD
2016-05-17 21:52:40,077 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:40,077 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:40,077 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0FE2DE5D6F974A9CBA974E77810B04FD
2016-05-17 21:52:40,084 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DB8E2F2045794DDEAF203EE1705A8C16, ordinal: 1
2016-05-17 21:52:40,084 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DB8E2F2045794DDEAF203EE1705A8C16, mem on start (mb), free: 419, total: 556, max: 672
2016-05-17 21:52:40,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:40,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:40,087 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:40,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0FE2DE5D6F974A9CBA974E77810B04FD
2016-05-17 21:52:40,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:40,088 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:40,088 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:40,089 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:40,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:40,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:40,092 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:40,092 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E50C8771203D4054B9211097074879AF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:40,092 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0FE2DE5D6F974A9CBA974E77810B04FD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:40,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:40,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,094 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:40,095 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:40,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E50C8771203D4054B9211097074879AF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:40,095 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E50C8771203D4054B9211097074879AF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:40,095 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:40,095 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:40,095 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:40,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:40,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:40,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:40,098 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,101 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521959775_0001_1_00_000000_0_10068, runDuration: 50942, ]
2016-05-17 21:52:40,102 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 21:52:40,103 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521959775_0001_1_00_000000_0_10068] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463521960103, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:52:40,103 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,103 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:40,103 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: E98B27387D99441FA4FC8D844CEF0506
2016-05-17 21:52:40,104 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E50C8771203D4054B9211097074879AF] #11 in 1ms
2016-05-17 21:52:40,104 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"], id: 0FE2DE5D6F974A9CBA974E77810B04FD
2016-05-17 21:52:40,104 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E50C8771203D4054B9211097074879AF] #11, status:false, isInterrupted:false
2016-05-17 21:52:40,105 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:40,105 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-17 21:52:40,106 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] E98B27387D99441FA4FC8D844CEF0506, for 1 inputs
2016-05-17 21:52:40,106 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 21:52:40,106 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:40,107 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:40,107 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 21:52:40,107 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,107 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:40,107 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DB8E2F2045794DDEAF203EE1705A8C16, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:52:40,109 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@177dd5b4; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@46592878
2016-05-17 21:52:40,113 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:40,114 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:40,114 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:40,114 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:40,114 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,115 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521959775_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:40,115 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,115 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DB8E2F2045794DDEAF203EE1705A8C16, mem on close (mb), free: 416, total: 556, max: 672
2016-05-17 21:52:40,116 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E50C8771203D4054B9211097074879AF
2016-05-17 21:52:40,116 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:40,117 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:40,117 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:40,117 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:40,117 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,118 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:40,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:40,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521960107, Process_Duration=8, Process_End_Time=1463521960115, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 21:52:40,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, taskAttemptId=attempt_1463521959775_0001_1_01_000000_0, startTime=1463521960067, finishTime=1463521960119, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960107, Process_Duration=8, Process_End_Time=1463521960115, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:40,120 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521959775_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:40,120 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:40,121 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:40,121 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,120 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521959775_0001_1_01_000000_0
2016-05-17 21:52:40,121 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521959775_0001_00_000002
2016-05-17 21:52:40,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:TASK_FINISHED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, taskId=task_1463521959775_0001_1_01_000000, startTime=1463521960067, finishTime=1463521960121, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463521959775_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960107, Process_Duration=8, Process_End_Time=1463521960115, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:40,122 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521959775_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:40,122 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] : 1
2016-05-17 21:52:40,122 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:40,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:40,123 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:40,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:VERTEX_FINISHED]: vertexName=DB8E2F2045794DDEAF203EE1705A8C16, vertexId=vertex_1463521959775_0001_1_01, initRequestedTime=1463521959950, initedTime=1463521959957, startRequestedTime=1463521959966, startedTime=1463521959966, finishTime=1463521960122, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=583532544, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960107, Process_Duration=8, Process_End_Time=1463521960115, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463521960067, firstTasksToStart=[ task_1463521959775_0001_1_01_000000 ], lastTaskFinishTime=1463521960119, lastTasksToFinish=[ task_1463521959775_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521959775_0001_1_01_000000 ], longestDurationTasks=[ task_1463521959775_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:40,124 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:40,124 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:40,124 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521959775_0001_00_000002, stoppedTime=1463521960124, exitStatus=0
2016-05-17 21:52:40,124 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:40,124 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:40,124 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521959775_0001_1
2016-05-17 21:52:40,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521959775_0001_1
2016-05-17 21:52:40,125 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521959775_0001_1_00 [E50C8771203D4054B9211097074879AF]
2016-05-17 21:52:40,125 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0FE2DE5D6F974A9CBA974E77810B04FD for vertex: vertex_1463521959775_0001_1_01 [DB8E2F2045794DDEAF203EE1705A8C16]
2016-05-17 21:52:40,124 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:40,131 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521959775_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:40,132 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521959775_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:40,132 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521959775_0001_00_000002, Exiting
2016-05-17 21:52:40,132 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521959775_0001_00_000002
2016-05-17 21:52:40,132 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521959775_0001_00_000002
2016-05-17 21:52:40,133 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521959775_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:40,133 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521959775_0001_00_000002 completed successfully
2016-05-17 21:52:40,135 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521959775_0001_1, queueSize=0
2016-05-17 21:52:40,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:40,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521959775_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521959775_0001_1, startTime=1463521959948, finishTime=1463521960133, timeTaken=185, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1167065088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043920139, Process_Duration=10, Process_End_Time=2927043920149, Read_Duration=3, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 21:52:40,137 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521959775_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:40,137 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521959775_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:40,138 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521959775_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:40,138 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521959775_0001_00_000002
2016-05-17 21:52:40 Completed Dag: dag_1463521959775_0001_1
2016-05-17 21:52:40,139 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521959775_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:40,140 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521959775_0001_1_post
2016-05-17 21:52:40,140 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,140 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:40,145 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463521959775_0001
2016-05-17 21:52:40,148 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:40,148 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:40,148 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:40,149 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:40,149 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:40,149 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:40,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:40,150 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:40,150 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:40,151 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:40,151 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:40,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:40,152 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:40,157 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5C52536865C45ADBA1E25160089FA68/C7D12E72530547F4B247093C6DC2E360/.tez/application_1463521959775_0001
2016-05-17 21:52:40,157 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2/_temporary
2016-05-17 21:52:40,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.401
2016-05-17 21:52:40,229 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-17 21:52:40,252 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:40,253 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:40,297 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:52:40,310 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-05-17 21:52:40,311 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:40,311 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:40,311 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:40,327 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 21:52:40,333 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-17 21:52:40,377 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 21:52:40,377 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,377 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-17 21:52:40,378 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-17 21:52:40,378 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-17 21:52:40,378 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-17 21:52:40,378 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount3/count3
2016-05-17 21:52:40,379 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:40,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 21:52:40,395 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:40,395 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:40,395 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:40,404 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001 doesn't exist and is created
2016-05-17 21:52:40,420 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:40,420 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:40,420 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001
2016-05-17 21:52:40,423 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521960395_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:40,433 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:40,433 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:40,434 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:40,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:40,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001/localmode-log-dir
2016-05-17 21:52:40,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001/localmode-log-dir/history.txt.appattempt_1463521960395_0001_000000, maxErrors=10
2016-05-17 21:52:40,435 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:40,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521960395_0001_000000, appSubmitTime=1463521960423, launchTime=1463521960423
2016-05-17 21:52:40,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:40,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:40,442 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:40,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:40,448 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48877
2016-05-17 21:52:40,449 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:40,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521960395_0001_000000, startTime=1463521960450
2016-05-17 21:52:40,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:40,451 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:40,520 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:40,521 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:40,521 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463521960395_0001, dagName=[8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3
2016-05-17 21:52:40,537 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3
2016-05-17 21:52:40,538 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521960395_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001/localmode-log-dir/dag_1463521960395_0001_1.dot
2016-05-17 21:52:40,539 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001/localmode-log-dir/dag_1463521960395_0001_1-tez-dag.pb.txt
2016-05-17 21:52:40,546 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521960395_0001_1
2016-05-17 21:52:40,546 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,547 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3
2016-05-17 21:52:40 Running Dag: dag_1463521960395_0001_1
2016-05-17 21:52:40,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521960395_0001_1, submitTime=1463521960537
2016-05-17 21:52:40,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:40,559 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,560 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:40,560 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:40,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521960395_0001_1, initTime=1463521960559
2016-05-17 21:52:40,560 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960395_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:40,561 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_STARTED]: dagID=dag_1463521960395_0001_1, startTime=1463521960561
2016-05-17 21:52:40,561 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:40,561 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960395_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:40,561 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:40,561 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D8187D446EE8478FA794302BE36A878E : {A0EAA328737C44C684F185EDE31B2477={InputName=A0EAA328737C44C684F185EDE31B2477}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:40,562 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A0EAA328737C44C684F185EDE31B2477, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:40,562 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,562 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,562 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,562 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]. Starting root input initializers: 1
2016-05-17 21:52:40,563 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:40,563 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,563 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A0EAA328737C44C684F185EDE31B2477 on vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,564 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463521960395_0001, dagName=[8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3
2016-05-17 21:52:40,565 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:40,565 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:40,566 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,566 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A0EAA328737C44C684F185EDE31B2477 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:40,566 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,567 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,567 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FD21EC6DAF884AFABDA16FD56BF6A583, vertexId=vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:40,567 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:40,568 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FD21EC6DAF884AFABDA16FD56BF6A583, vertexId=vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,568 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:40,570 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 21:52:40,570 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FD21EC6DAF884AFABDA16FD56BF6A583, vertexName=A58EAD1A40BE475786E3B211FCAC832D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:40,571 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FD21EC6DAF884AFABDA16FD56BF6A583, vertexId=vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,570 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A0EAA328737C44C684F185EDE31B2477 on vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, vertexId=vertex_1463521960395_0001_1_01, initRequestedTime=1463521960563, initedTime=1463521960572, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:40,572 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:40,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:40,573 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:40,573 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A0EAA328737C44C684F185EDE31B2477=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:40,574 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] parallelism set to 1
2016-05-17 21:52:40,574 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:40,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521960395_0001_1_00, eventCount=1
2016-05-17 21:52:40,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D8187D446EE8478FA794302BE36A878E, vertexId=vertex_1463521960395_0001_1_00, initRequestedTime=1463521960561, initedTime=1463521960575, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:40,576 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] with distanceFromRoot: 0
2016-05-17 21:52:40,576 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:40,576 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:40,576 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,577 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:40,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521960395_0001_1_00, startRequestedTime=1463521960576, startedTime=1463521960576
2016-05-17 21:52:40,578 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:40,578 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:40,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_STARTED]: vertexName=D8187D446EE8478FA794302BE36A878E, taskId=task_1463521960395_0001_1_00_000000, scheduledTime=1463521960578, launchTime=1463521960578
2016-05-17 21:52:40,579 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:40,579 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521960395_0001_1_00 for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] numStartedSources: 1 numSources: 1
2016-05-17 21:52:40,579 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] with distanceFromRoot: 1
2016-05-17 21:52:40,580 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,580 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D8187D446EE8478FA794302BE36A878E in vertex: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,581 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A58EAD1A40BE475786E3B211FCAC832D with 1 source tasks and 1 pending tasks
2016-05-17 21:52:40,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:40,581 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521960395_0001_1_01, startRequestedTime=1463521960580, startedTime=1463521960580
2016-05-17 21:52:40,582 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:40,582 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3, VertexName: D8187D446EE8478FA794302BE36A878E, VertexParallelism: 1, TaskAttemptID:attempt_1463521960395_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A0EAA328737C44C684F185EDE31B2477, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A58EAD1A40BE475786E3B211FCAC832D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:40,583 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:40,583 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:40,584 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:40,585 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:40,585 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:40,586 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521960395_0001_00_000001
2016-05-17 21:52:40,586 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:40,588 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521960395_0001_00_000001
2016-05-17 21:52:40,588 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:40,588 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463521960395_0001, with dag id: dag_1463521960395_0001_1
2016-05-17 21:52:40,590 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:40,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:40,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521960395_0001_00_000001, launchTime=1463521960589
2016-05-17 21:52:40,593 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960395_0001_00_000001
2016-05-17 21:52:40,593 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521960395_0001_1_00_000000_0] to container: [container_1463521960395_0001_00_000001]
2016-05-17 21:52:40,593 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:40,594 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521960395_0001_1_00_000000_0] started. Is using containerId: [container_1463521960395_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:40,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D8187D446EE8478FA794302BE36A878E, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0, startTime=1463521960594, containerId=container_1463521960395_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521960395_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:40,594 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:40,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:40,594 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:40,595 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521960395_0001_00_000001 given task: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,595 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960395_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40 Starting to run new task attempt: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,596 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,596 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,596 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:40,597 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:40,597 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:40,597 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:40,598 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:40,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3, VertexName: D8187D446EE8478FA794302BE36A878E, VertexParallelism: 1, TaskAttemptID:attempt_1463521960395_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A0EAA328737C44C684F185EDE31B2477, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A58EAD1A40BE475786E3B211FCAC832D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:40,598 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:40,599 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:40,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,599 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521960395_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:40,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A0EAA328737C44C684F185EDE31B2477, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:40,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:40,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,603 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,604 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:40,605 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,605 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:40,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A58EAD1A40BE475786E3B211FCAC832D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:40,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:40,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,608 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:40,608 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,612 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D8187D446EE8478FA794302BE36A878E, ordinal: 0
2016-05-17 21:52:40,612 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8187D446EE8478FA794302BE36A878E, mem on start (mb), free: 325, total: 556, max: 672
2016-05-17 21:52:40,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:40,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:40,613 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:40,613 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:40,613 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:40,614 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:40,614 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:40,614 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:40,614 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:40,615 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A0EAA328737C44C684F185EDE31B2477, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:40,615 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A58EAD1A40BE475786E3B211FCAC832D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:40,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:40,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A0EAA328737C44C684F185EDE31B2477 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:40,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,616 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:40,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:40,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:40,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:40,618 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,619 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:40,619 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1de4b844
2016-05-17 21:52:40,620 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1de4b844
2016-05-17 21:52:40,620 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:40,621 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:40,621 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,622 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: A0EAA328737C44C684F185EDE31B2477
2016-05-17 21:52:40,622 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 0FC613E742CE4266A24609E9E8BE9972
2016-05-17 21:52:40,622 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 0FC613E742CE4266A24609E9E8BE9972
2016-05-17 21:52:40,624 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@748860cc; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@258b2cd9; comparator=cascading.tuple.hadoop.util.TupleComparator@542d6dd0; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:40,624 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:40,624 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:40,624 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:40,643 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:40,643 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:40,643 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:40,643 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:40,644 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:40,645 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,646 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:40,646 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D8187D446EE8478FA794302BE36A878E, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:40,648 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,649 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8187D446EE8478FA794302BE36A878E, mem on close (mb), free: 417, total: 548, max: 672
2016-05-17 21:52:40,649 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:40,649 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:40,649 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 21:52:40,649 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:52:40,653 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,656 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:40,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521960395_0001_1_00, eventCount=1
2016-05-17 21:52:40,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521960646, Process_Duration=2, Process_End_Time=1463521960648, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:52:40,657 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:40,657 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,657 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:40,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D8187D446EE8478FA794302BE36A878E, taskAttemptId=attempt_1463521960395_0001_1_00_000000_0, startTime=1463521960594, finishTime=1463521960657, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960646, Process_Duration=2, Process_End_Time=1463521960648, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:40,658 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:40,658 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:40,657 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960395_0001_00_000001
2016-05-17 21:52:40,658 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521960395_0001_1_00_000000_0
2016-05-17 21:52:40,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:40,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_FINISHED]: vertexName=D8187D446EE8478FA794302BE36A878E, taskId=task_1463521960395_0001_1_00_000000, startTime=1463521960594, finishTime=1463521960659, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1463521960395_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960646, Process_Duration=2, Process_End_Time=1463521960648, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:40,660 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:40,660 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] : 1
2016-05-17 21:52:40,660 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,661 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:40,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_FINISHED]: vertexName=D8187D446EE8478FA794302BE36A878E, vertexId=vertex_1463521960395_0001_1_00, initRequestedTime=1463521960561, initedTime=1463521960575, startRequestedTime=1463521960576, startedTime=1463521960576, finishTime=1463521960660, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960646, Process_Duration=2, Process_End_Time=1463521960648, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463521960594, firstTasksToStart=[ task_1463521960395_0001_1_00_000000 ], lastTaskFinishTime=1463521960657, lastTasksToFinish=[ task_1463521960395_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521960395_0001_1_00_000000 ], longestDurationTasks=[ task_1463521960395_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:40,661 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:40,661 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:40,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521960395_0001_00_000001, stoppedTime=1463521960662, exitStatus=0
2016-05-17 21:52:40,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:40,662 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:40,662 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] attempt: attempt_1463521960395_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:40,662 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A58EAD1A40BE475786E3B211FCAC832D
2016-05-17 21:52:40,662 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,663 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:40,663 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521960395_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:40,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_STARTED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, taskId=task_1463521960395_0001_1_01_000000, scheduledTime=1463521960663, launchTime=1463521960663
2016-05-17 21:52:40,663 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:40,663 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:40,663 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:40,664 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,664 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3, VertexName: A58EAD1A40BE475786E3B211FCAC832D, VertexParallelism: 1, TaskAttemptID:attempt_1463521960395_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D8187D446EE8478FA794302BE36A878E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FD21EC6DAF884AFABDA16FD56BF6A583, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:40,664 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:40,664 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:40,665 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521960395_0001_00_000002
2016-05-17 21:52:40,665 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:40,666 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521960395_0001_00_000002
2016-05-17 21:52:40,666 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:40,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:40,667 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:40,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521960395_0001_00_000002, launchTime=1463521960667
2016-05-17 21:52:40,668 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960395_0001_00_000002
2016-05-17 21:52:40,668 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521960395_0001_1_01_000000_0] to container: [container_1463521960395_0001_00_000002]
2016-05-17 21:52:40,668 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:40,669 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521960395_0001_00_000002 given task: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,669 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960395_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,669 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521960395_0001_1_01_000000_0] started. Is using containerId: [container_1463521960395_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:40 Starting to run new task attempt: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,669 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,669 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,670 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:40,670 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:40,670 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:40,670 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:40,670 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:40,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F] count/(1/1) ...latform/testCount3/count3, VertexName: A58EAD1A40BE475786E3B211FCAC832D, VertexParallelism: 1, TaskAttemptID:attempt_1463521960395_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D8187D446EE8478FA794302BE36A878E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FD21EC6DAF884AFABDA16FD56BF6A583, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:40,671 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:40,671 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521960395_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:40,671 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960395_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:40,672 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521960395_0001_00_000001, Exiting
2016-05-17 21:52:40,672 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521960395_0001_00_000001
2016-05-17 21:52:40,672 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521960395_0001_00_000001
2016-05-17 21:52:40,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0, startTime=1463521960669, containerId=container_1463521960395_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521960395_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:40,670 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:40,674 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:40,672 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:40,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,672 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521960395_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:40,676 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:40,675 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521960395_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:40,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FD21EC6DAF884AFABDA16FD56BF6A583, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:40,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:40,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FD21EC6DAF884AFABDA16FD56BF6A583
2016-05-17 21:52:40,677 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:40,678 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FD21EC6DAF884AFABDA16FD56BF6A583
2016-05-17 21:52:40,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D8187D446EE8478FA794302BE36A878E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:40,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:40,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,679 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521960395_0001_00_000001 completed successfully
2016-05-17 21:52:40,679 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:40,681 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:40,682 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:40,682 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,684 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:40,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,692 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:40,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FD21EC6DAF884AFABDA16FD56BF6A583
2016-05-17 21:52:40,693 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A58EAD1A40BE475786E3B211FCAC832D, ordinal: 1
2016-05-17 21:52:40,693 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A58EAD1A40BE475786E3B211FCAC832D, mem on start (mb), free: 410, total: 548, max: 672
2016-05-17 21:52:40,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:40,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:40,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:40,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:40,695 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:40,696 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:40,696 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:40,696 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:40,697 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:40,697 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:40,698 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FD21EC6DAF884AFABDA16FD56BF6A583, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:40,698 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D8187D446EE8478FA794302BE36A878E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:40,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:40,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,700 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:40,700 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:40,700 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D8187D446EE8478FA794302BE36A878E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:40,700 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:40,700 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:40,701 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:40,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D8187D446EE8478FA794302BE36A878E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:40,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:40,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:40,705 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:40,705 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,705 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521960395_0001_1_00_000000_0_10074, runDuration: 47821, ]
2016-05-17 21:52:40,707 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521960395_0001_1_00_000000_0_10074] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463521960707, TimeTaken=1, Rate=0.44 MB/s
2016-05-17 21:52:40,707 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,707 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:40,708 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D8187D446EE8478FA794302BE36A878E] #12 in 2ms
2016-05-17 21:52:40,708 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D8187D446EE8478FA794302BE36A878E] #12, status:false, isInterrupted:false
2016-05-17 21:52:40,709 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:40,709 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 21:52:40,710 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:40,710 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:40,710 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 21:52:40,710 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,710 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:40,713 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 21:52:40,713 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 0FC613E742CE4266A24609E9E8BE9972
2016-05-17 21:52:40,714 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"], id: FD21EC6DAF884AFABDA16FD56BF6A583
2016-05-17 21:52:40,714 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-17 21:52:40,714 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 0FC613E742CE4266A24609E9E8BE9972, for 1 inputs
2016-05-17 21:52:40,715 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A58EAD1A40BE475786E3B211FCAC832D, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:40,717 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@29ef44ef; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@1b96bc6b
2016-05-17 21:52:40,720 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:40,721 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:40,723 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,723 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521960395_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:40,724 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,724 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A58EAD1A40BE475786E3B211FCAC832D, mem on close (mb), free: 408, total: 548, max: 672
2016-05-17 21:52:40,725 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D8187D446EE8478FA794302BE36A878E
2016-05-17 21:52:40,725 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:40,726 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:40,726 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:40,726 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:40,727 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:40 Completed running task attempt: attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,728 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:40,729 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:40,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, taskAttemptId=attempt_1463521960395_0001_1_01_000000_0, startTime=1463521960669, finishTime=1463521960729, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960715, Process_Duration=9, Process_End_Time=1463521960724, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:40,730 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960395_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:40,730 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521960395_0001_1_01_000000_0
2016-05-17 21:52:40,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:TASK_FINISHED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, taskId=task_1463521960395_0001_1_01_000000, startTime=1463521960669, finishTime=1463521960730, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463521960395_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960715, Process_Duration=9, Process_End_Time=1463521960724, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:40,731 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960395_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:40,731 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] : 1
2016-05-17 21:52:40,731 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,731 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:40,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:40,732 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:40,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521960715, Process_Duration=9, Process_End_Time=1463521960724, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 21:52:40,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:VERTEX_FINISHED]: vertexName=A58EAD1A40BE475786E3B211FCAC832D, vertexId=vertex_1463521960395_0001_1_01, initRequestedTime=1463521960563, initedTime=1463521960572, startRequestedTime=1463521960580, startedTime=1463521960580, finishTime=1463521960731, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=574619648, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521960715, Process_Duration=9, Process_End_Time=1463521960724, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463521960669, firstTasksToStart=[ task_1463521960395_0001_1_01_000000 ], lastTaskFinishTime=1463521960729, lastTasksToFinish=[ task_1463521960395_0001_1_01_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521960395_0001_1_01_000000 ], longestDurationTasks=[ task_1463521960395_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:40,733 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:40,733 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:40,733 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:40,734 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,734 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521960395_0001_00_000002, stoppedTime=1463521960734, exitStatus=0
2016-05-17 21:52:40,734 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960395_0001_00_000002
2016-05-17 21:52:40,734 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:40,734 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521960395_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:40,734 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:40,734 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960395_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:40,734 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:40,735 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521960395_0001_00_000002, Exiting
2016-05-17 21:52:40,735 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521960395_0001_00_000002
2016-05-17 21:52:40,735 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521960395_0001_00_000002
2016-05-17 21:52:40,735 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521960395_0001_1
2016-05-17 21:52:40,735 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521960395_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:40,735 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521960395_0001_00_000002 completed successfully
2016-05-17 21:52:40,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521960395_0001_1
2016-05-17 21:52:40,736 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521960395_0001_1_00 [D8187D446EE8478FA794302BE36A878E]
2016-05-17 21:52:40,736 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: FD21EC6DAF884AFABDA16FD56BF6A583 for vertex: vertex_1463521960395_0001_1_01 [A58EAD1A40BE475786E3B211FCAC832D]
2016-05-17 21:52:40,746 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521960395_0001_1, queueSize=0
2016-05-17 21:52:40,746 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:40,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960395_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521960395_0001_1, startTime=1463521960561, finishTime=1463521960744, timeTaken=183, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=5, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1149239296, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043921361, Process_Duration=11, Process_End_Time=2927043921372, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 21:52:40,746 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521960395_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:40,746 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960395_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:40,746 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521960395_0001_00_000002
2016-05-17 21:52:40,746 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960395_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:40 Completed Dag: dag_1463521960395_0001_1
2016-05-17 21:52:40,747 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521960395_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:40,747 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521960395_0001_1_post
2016-05-17 21:52:40,748 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:40,748 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:40,753 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463521960395_0001
2016-05-17 21:52:40,754 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:40,755 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:40,755 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:40,755 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:40,755 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:40,756 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:40,756 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:40,756 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:40,756 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:40,757 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:40,757 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:40,757 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:40,757 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:40,762 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8D76F5913C5849A5911875CA8006B384/DB15F13A5CD94007BEADF0638B42361F/.tez/application_1463521960395_0001
2016-05-17 21:52:40,763 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3/_temporary
2016-05-17 21:52:40,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.385
2016-05-17 21:52:40,816 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:40,816 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:40,831 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-17 21:52:40,854 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:40,855 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:40,887 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:52:40,895 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 21:52:40,896 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:40,896 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:40,896 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:40,910 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 21:52:40,916 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-17 21:52:40,959 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 21:52:40,959 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:40,960 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-17 21:52:40,960 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-17 21:52:40,960 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-17 21:52:40,960 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-17 21:52:40,961 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount4/count4
2016-05-17 21:52:40,961 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:40,973 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 21:52:40,977 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:40,978 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:40,978 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:40,987 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001 doesn't exist and is created
2016-05-17 21:52:41,003 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:41,003 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:41,004 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001
2016-05-17 21:52:41,006 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521960977_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:41,016 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:41,017 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:41,018 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:41,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:41,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001/localmode-log-dir
2016-05-17 21:52:41,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001/localmode-log-dir/history.txt.appattempt_1463521960977_0001_000000, maxErrors=10
2016-05-17 21:52:41,019 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:41,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521960977_0001_000000, appSubmitTime=1463521961006, launchTime=1463521961006
2016-05-17 21:52:41,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:41,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:41,029 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58479
2016-05-17 21:52:41,029 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:41,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:41,032 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:41,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521960977_0001_000000, startTime=1463521961034
2016-05-17 21:52:41,034 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:41,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:41,104 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:41,104 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:41,104 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463521960977_0001, dagName=[52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4
2016-05-17 21:52:41,120 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4
2016-05-17 21:52:41,121 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521960977_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001/localmode-log-dir/dag_1463521960977_0001_1.dot
2016-05-17 21:52:41,122 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001/localmode-log-dir/dag_1463521960977_0001_1-tez-dag.pb.txt
2016-05-17 21:52:41,130 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521960977_0001_1
2016-05-17 21:52:41,130 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,130 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4
2016-05-17 21:52:41 Running Dag: dag_1463521960977_0001_1
2016-05-17 21:52:41,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521960977_0001_1, submitTime=1463521961120
2016-05-17 21:52:41,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:41,142 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,142 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:41,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:41,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521960977_0001_1, initTime=1463521961141
2016-05-17 21:52:41,143 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960977_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:41,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_STARTED]: dagID=dag_1463521960977_0001_1, startTime=1463521961143
2016-05-17 21:52:41,143 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:41,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:41,144 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960977_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:41,144 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EC8306BF07FC4B23939584C6A74A43C0 : {BD188B76DC47455FB30B8AEB61242335={InputName=BD188B76DC47455FB30B8AEB61242335}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:41,144 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BD188B76DC47455FB30B8AEB61242335, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:41,144 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463521960977_0001, dagName=[52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4
2016-05-17 21:52:41,144 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,145 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,145 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,145 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]. Starting root input initializers: 1
2016-05-17 21:52:41,146 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:41,146 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BD188B76DC47455FB30B8AEB61242335 on vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,146 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,148 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:41,149 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BD188B76DC47455FB30B8AEB61242335 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:41,149 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:41,149 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:41,150 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,150 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,150 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,151 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, vertexId=vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:41,151 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, vertexId=vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,150 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:41,153 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:41,153 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, vertexId=vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,154 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 21:52:41,155 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BD188B76DC47455FB30B8AEB61242335 on vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,155 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:41,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, vertexId=vertex_1463521960977_0001_1_01, initRequestedTime=1463521961146, initedTime=1463521961155, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:41,156 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:41,156 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:41,156 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BD188B76DC47455FB30B8AEB61242335=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:41,156 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] parallelism set to 1
2016-05-17 21:52:41,156 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:41,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521960977_0001_1_00, eventCount=1
2016-05-17 21:52:41,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:41,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, vertexId=vertex_1463521960977_0001_1_00, initRequestedTime=1463521961144, initedTime=1463521961157, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:41,157 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] with distanceFromRoot: 0
2016-05-17 21:52:41,157 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:41,158 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,158 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:41,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521960977_0001_1_00, startRequestedTime=1463521961158, startedTime=1463521961158
2016-05-17 21:52:41,158 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:41,159 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:41,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_STARTED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, taskId=task_1463521960977_0001_1_00_000000, scheduledTime=1463521961159, launchTime=1463521961159
2016-05-17 21:52:41,159 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:41,159 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521960977_0001_1_00 for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] numStartedSources: 1 numSources: 1
2016-05-17 21:52:41,159 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] with distanceFromRoot: 1
2016-05-17 21:52:41,160 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,160 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EC8306BF07FC4B23939584C6A74A43C0 in vertex: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,160 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B3A5359E5D9546B7B4AD5BD3BBA1F66F with 1 source tasks and 1 pending tasks
2016-05-17 21:52:41,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:41,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521960977_0001_1_01, startRequestedTime=1463521961160, startedTime=1463521961160
2016-05-17 21:52:41,161 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:41,161 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4, VertexName: EC8306BF07FC4B23939584C6A74A43C0, VertexParallelism: 1, TaskAttemptID:attempt_1463521960977_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD188B76DC47455FB30B8AEB61242335, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:41,162 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:41,162 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:41,162 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:41,163 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:41,163 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:41,164 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521960977_0001_00_000001
2016-05-17 21:52:41,164 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:41,166 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521960977_0001_00_000001
2016-05-17 21:52:41,166 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:41,167 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:41,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:41,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521960977_0001_00_000001, launchTime=1463521961167
2016-05-17 21:52:41,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463521960977_0001, with dag id: dag_1463521960977_0001_1
2016-05-17 21:52:41,170 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960977_0001_00_000001
2016-05-17 21:52:41,171 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521960977_0001_1_00_000000_0] to container: [container_1463521960977_0001_00_000001]
2016-05-17 21:52:41,171 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:41,171 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521960977_0001_00_000001 given task: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,171 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960977_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,174 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521960977_0001_1_00_000000_0] started. Is using containerId: [container_1463521960977_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:41,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0, startTime=1463521961172, containerId=container_1463521960977_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521960977_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:41,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:41,175 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:41,175 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:41 Starting to run new task attempt: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,176 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,176 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,176 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:41,176 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:41,176 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:41,176 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:41,178 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:41,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4, VertexName: EC8306BF07FC4B23939584C6A74A43C0, VertexParallelism: 1, TaskAttemptID:attempt_1463521960977_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD188B76DC47455FB30B8AEB61242335, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:41,179 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:41,180 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,180 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:41,180 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,181 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521960977_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:41,181 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BD188B76DC47455FB30B8AEB61242335, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:41,181 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:41,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:41,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:41,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,182 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,184 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:41,186 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:41,186 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,189 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:41,189 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,190 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EC8306BF07FC4B23939584C6A74A43C0, ordinal: 0
2016-05-17 21:52:41,190 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EC8306BF07FC4B23939584C6A74A43C0, mem on start (mb), free: 318, total: 548, max: 672
2016-05-17 21:52:41,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:41,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:41,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:41,191 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:41,192 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:41,192 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:41,192 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:41,192 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:41,192 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:41,193 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BD188B76DC47455FB30B8AEB61242335, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:41,193 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B3A5359E5D9546B7B4AD5BD3BBA1F66F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:41,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:41,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BD188B76DC47455FB30B8AEB61242335 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:41,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,193 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:41,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:41,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:41,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:41,194 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,194 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:41,195 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@73e8f917
2016-05-17 21:52:41,196 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@73e8f917
2016-05-17 21:52:41,196 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:41,196 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:41,198 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:41,198 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: BD188B76DC47455FB30B8AEB61242335
2016-05-17 21:52:41,198 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 2EE1A62750F64CE0888F2B9991F60303
2016-05-17 21:52:41,199 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 2EE1A62750F64CE0888F2B9991F60303
2016-05-17 21:52:41,201 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@21a7333c; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6814b966; comparator=cascading.tuple.hadoop.util.TupleComparator@28980cec; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:41,202 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:41,202 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:41,202 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:41,223 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:41,223 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:41,224 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:41,224 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:41,224 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:41,224 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 21:52:41,224 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:41,225 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EC8306BF07FC4B23939584C6A74A43C0, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:41,228 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,228 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EC8306BF07FC4B23939584C6A74A43C0, mem on close (mb), free: 423, total: 555, max: 672
2016-05-17 21:52:41,229 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:41,229 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:41,229 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 21:52:41,229 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:52:41,233 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:41 Completed running task attempt: attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,236 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:41,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521960977_0001_1_00, eventCount=1
2016-05-17 21:52:41,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:41,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, taskAttemptId=attempt_1463521960977_0001_1_00_000000_0, startTime=1463521961172, finishTime=1463521961237, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961225, Process_Duration=3, Process_End_Time=1463521961228, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:41,237 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:41,238 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521960977_0001_1_00_000000_0
2016-05-17 21:52:41,238 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:41,238 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:41,238 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_FINISHED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, taskId=task_1463521960977_0001_1_00_000000, startTime=1463521961172, finishTime=1463521961238, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1463521960977_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961225, Process_Duration=3, Process_End_Time=1463521961228, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:52:41,238 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:41,239 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] : 1
2016-05-17 21:52:41,239 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:41,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_FINISHED]: vertexName=EC8306BF07FC4B23939584C6A74A43C0, vertexId=vertex_1463521960977_0001_1_00, initRequestedTime=1463521961144, initedTime=1463521961157, startRequestedTime=1463521961158, startedTime=1463521961158, finishTime=1463521961239, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961225, Process_Duration=3, Process_End_Time=1463521961228, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463521961172, firstTasksToStart=[ task_1463521960977_0001_1_00_000000 ], lastTaskFinishTime=1463521961237, lastTasksToFinish=[ task_1463521960977_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521960977_0001_1_00_000000 ], longestDurationTasks=[ task_1463521960977_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:41,240 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:41,240 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:41,241 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] attempt: attempt_1463521960977_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:41,241 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B3A5359E5D9546B7B4AD5BD3BBA1F66F
2016-05-17 21:52:41,241 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521960977_0001_00_000001, stoppedTime=1463521961241, exitStatus=0
2016-05-17 21:52:41,241 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:41,241 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:41,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:41,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_STARTED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, taskId=task_1463521960977_0001_1_01_000000, scheduledTime=1463521961242, launchTime=1463521961242
2016-05-17 21:52:41,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:41,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521961225, Process_Duration=3, Process_End_Time=1463521961228, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:52:41,242 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:41,243 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521960977_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:41,243 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:41,243 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,243 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,243 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:41,243 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4, VertexName: B3A5359E5D9546B7B4AD5BD3BBA1F66F, VertexParallelism: 1, TaskAttemptID:attempt_1463521960977_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EC8306BF07FC4B23939584C6A74A43C0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:41,244 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:41,244 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:41,244 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960977_0001_00_000001
2016-05-17 21:52:41,244 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521960977_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:41,245 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960977_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:41,245 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521960977_0001_00_000002
2016-05-17 21:52:41,245 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:41,246 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521960977_0001_00_000001, Exiting
2016-05-17 21:52:41,246 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521960977_0001_00_000001
2016-05-17 21:52:41,246 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521960977_0001_00_000001
2016-05-17 21:52:41,247 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521960977_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:41,247 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521960977_0001_00_000002
2016-05-17 21:52:41,247 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521960977_0001_00_000001 completed successfully
2016-05-17 21:52:41,248 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:41,248 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:52:41,250 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:41,250 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:41,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521960977_0001_00_000002, launchTime=1463521961250
2016-05-17 21:52:41,251 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960977_0001_00_000002
2016-05-17 21:52:41,252 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521960977_0001_1_01_000000_0] to container: [container_1463521960977_0001_00_000002]
2016-05-17 21:52:41,252 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:41,252 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521960977_0001_00_000002 given task: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,253 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960977_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,253 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521960977_0001_1_01_000000_0] started. Is using containerId: [container_1463521960977_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:41,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:41,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0, startTime=1463521961252, containerId=container_1463521960977_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521960977_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:41,253 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:41,254 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:41 Starting to run new task attempt: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,254 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,255 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,255 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:41,255 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:41,255 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:41,256 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:41,256 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:41,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51] count/(1/1) ...latform/testCount4/count4, VertexName: B3A5359E5D9546B7B4AD5BD3BBA1F66F, VertexParallelism: 1, TaskAttemptID:attempt_1463521960977_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EC8306BF07FC4B23939584C6A74A43C0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:41,256 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:41,257 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:41,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EC8306BF07FC4B23939584C6A74A43C0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:41,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:41,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,260 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:41,260 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:41,260 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,261 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521960977_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:41,263 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:41,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=25FDD7E0D74A4B7D90BB4AB1B8B6D84F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:41,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:41,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F
2016-05-17 21:52:41,266 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:41,266 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F
2016-05-17 21:52:41,270 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B3A5359E5D9546B7B4AD5BD3BBA1F66F, ordinal: 1
2016-05-17 21:52:41,270 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3A5359E5D9546B7B4AD5BD3BBA1F66F, mem on start (mb), free: 417, total: 555, max: 672
2016-05-17 21:52:41,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:41,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:41,276 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:41,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F
2016-05-17 21:52:41,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:41,277 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:41,277 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:41,278 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:41,278 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:41,278 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:41,278 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:41,279 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EC8306BF07FC4B23939584C6A74A43C0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:41,279 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 25FDD7E0D74A4B7D90BB4AB1B8B6D84F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:41,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:41,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EC8306BF07FC4B23939584C6A74A43C0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:41,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:41,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:41,281 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:41,281 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:41,282 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EC8306BF07FC4B23939584C6A74A43C0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:41,282 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:41,282 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:41,282 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:41,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:41,284 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,287 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 21:52:41,287 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521960977_0001_1_00_000000_0_10080, runDuration: 47675, ]
2016-05-17 21:52:41,287 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 2EE1A62750F64CE0888F2B9991F60303
2016-05-17 21:52:41,288 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"], id: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F
2016-05-17 21:52:41,288 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-17 21:52:41,289 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 2EE1A62750F64CE0888F2B9991F60303, for 1 inputs
2016-05-17 21:52:41,289 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521960977_0001_1_00_000000_0_10080] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463521961289, TimeTaken=1, Rate=0.44 MB/s
2016-05-17 21:52:41,290 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,290 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:41,290 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [EC8306BF07FC4B23939584C6A74A43C0] #13 in 2ms
2016-05-17 21:52:41,290 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EC8306BF07FC4B23939584C6A74A43C0] #13, status:false, isInterrupted:false
2016-05-17 21:52:41,291 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:41,292 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 21:52:41,292 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:41,292 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:41,292 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 21:52:41,293 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,293 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B3A5359E5D9546B7B4AD5BD3BBA1F66F, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:52:41,293 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:41,294 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1c20e05c; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@3269c671
2016-05-17 21:52:41,297 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:41,297 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:41,298 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,298 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521960977_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:41,298 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,299 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3A5359E5D9546B7B4AD5BD3BBA1F66F, mem on close (mb), free: 414, total: 555, max: 672
2016-05-17 21:52:41,299 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EC8306BF07FC4B23939584C6A74A43C0
2016-05-17 21:52:41,299 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:41,300 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:41,300 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:41,300 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:41,300 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:41 Completed running task attempt: attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,301 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:41,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521961293, Process_Duration=5, Process_End_Time=1463521961298, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 21:52:41,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, taskAttemptId=attempt_1463521960977_0001_1_01_000000_0, startTime=1463521961252, finishTime=1463521961302, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961293, Process_Duration=5, Process_End_Time=1463521961298, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:41,303 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521960977_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:41,303 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:41,303 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521960977_0001_1_01_000000_0
2016-05-17 21:52:41,302 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:41,303 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:41,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:41,304 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,304 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521960977_0001_00_000002
2016-05-17 21:52:41,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:TASK_FINISHED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, taskId=task_1463521960977_0001_1_01_000000, startTime=1463521961252, finishTime=1463521961303, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463521960977_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961293, Process_Duration=5, Process_End_Time=1463521961298, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 21:52:41,305 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521960977_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:41,305 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] : 1
2016-05-17 21:52:41,305 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:41,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:VERTEX_FINISHED]: vertexName=B3A5359E5D9546B7B4AD5BD3BBA1F66F, vertexId=vertex_1463521960977_0001_1_01, initRequestedTime=1463521961146, initedTime=1463521961155, startRequestedTime=1463521961160, startedTime=1463521961160, finishTime=1463521961305, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521961293, Process_Duration=5, Process_End_Time=1463521961298, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463521961252, firstTasksToStart=[ task_1463521960977_0001_1_01_000000 ], lastTaskFinishTime=1463521961302, lastTasksToFinish=[ task_1463521960977_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521960977_0001_1_01_000000 ], longestDurationTasks=[ task_1463521960977_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:41,307 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:41,307 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:41,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521960977_0001_00_000002, stoppedTime=1463521961307, exitStatus=0
2016-05-17 21:52:41,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:41,307 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:41,308 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:41,308 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521960977_0001_1
2016-05-17 21:52:41,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521960977_0001_1
2016-05-17 21:52:41,308 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 25FDD7E0D74A4B7D90BB4AB1B8B6D84F for vertex: vertex_1463521960977_0001_1_01 [B3A5359E5D9546B7B4AD5BD3BBA1F66F]
2016-05-17 21:52:41,315 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521960977_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:41,315 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521960977_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:41,315 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521960977_0001_00_000002, Exiting
2016-05-17 21:52:41,315 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521960977_0001_00_000002
2016-05-17 21:52:41,315 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521960977_0001_00_000002
2016-05-17 21:52:41,316 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521960977_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:41,316 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521960977_0001_00_000002 completed successfully
2016-05-17 21:52:41,316 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521960977_0001_1_00 [EC8306BF07FC4B23939584C6A74A43C0]
2016-05-17 21:52:41,319 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521960977_0001_1, queueSize=0
2016-05-17 21:52:41,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:41,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521960977_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521960977_0001_1, startTime=1463521961143, finishTime=1463521961316, timeTaken=173, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1164967936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043922518, Process_Duration=8, Process_End_Time=2927043922526, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 21:52:41,320 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521960977_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:41,320 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521960977_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:41,320 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521960977_0001_00_000002
2016-05-17 21:52:41,320 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521960977_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:41 Completed Dag: dag_1463521960977_0001_1
2016-05-17 21:52:41,320 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521960977_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:41,320 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521960977_0001_1_post
2016-05-17 21:52:41,321 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,321 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:41,326 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463521960977_0001
2016-05-17 21:52:41,327 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:41,328 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:41,328 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:41,328 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:41,328 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:41,329 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:41,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:41,329 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:41,329 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:41,329 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:41,330 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:41,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:41,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:41,334 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/52E0F87B81C9449B97AD3AB9205EBEE0/B5A1394F0B1E480AA4BB19C5EEA77E51/.tez/application_1463521960977_0001
2016-05-17 21:52:41,334 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4/_temporary
2016-05-17 21:52:41,335 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.375
2016-05-17 21:52:41,404 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-17 21:52:41,430 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:41,430 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:41,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:52:41,467 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:52:41,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:41,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:52:41,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:41,482 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:41,483 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-17 21:52:41,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:41,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:41,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-17 21:52:41,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:41,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:41,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:41,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testFilter/filter
2016-05-17 21:52:41,493 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:41,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:41,509 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:41,509 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:41,510 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:41,519 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001 doesn't exist and is created
2016-05-17 21:52:41,535 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:41,535 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:41,536 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001
2016-05-17 21:52:41,538 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521961509_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:41,548 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:41,548 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:41,549 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:41,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:41,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001/localmode-log-dir
2016-05-17 21:52:41,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001/localmode-log-dir/history.txt.appattempt_1463521961509_0001_000000, maxErrors=10
2016-05-17 21:52:41,550 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:41,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521961509_0001_000000, appSubmitTime=1463521961538, launchTime=1463521961538
2016-05-17 21:52:41,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:41,556 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:41,557 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54164
2016-05-17 21:52:41,558 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:41,558 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:41,559 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:41,561 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521961509_0001_000000, startTime=1463521961561
2016-05-17 21:52:41,561 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:41,561 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:41,635 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:41,635 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:41,636 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463521961509_0001, dagName=[7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter
2016-05-17 21:52:41,654 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter
2016-05-17 21:52:41,655 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521961509_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001/localmode-log-dir/dag_1463521961509_0001_1.dot
2016-05-17 21:52:41,655 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001/localmode-log-dir/dag_1463521961509_0001_1-tez-dag.pb.txt
2016-05-17 21:52:41,664 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521961509_0001_1
2016-05-17 21:52:41,665 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,665 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter
2016-05-17 21:52:41 Running Dag: dag_1463521961509_0001_1
2016-05-17 21:52:41,680 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:41,680 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:41,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521961509_0001_1, submitTime=1463521961654
2016-05-17 21:52:41,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:41,682 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CDED5065C842484FB462A32D3167D312
2016-05-17 21:52:41,682 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:41,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521961509_0001_1, initTime=1463521961681
2016-05-17 21:52:41,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:41,683 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961509_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:41,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_STARTED]: dagID=dag_1463521961509_0001_1, startTime=1463521961683
2016-05-17 21:52:41,683 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:41,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:41,684 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961509_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:41,684 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CDED5065C842484FB462A32D3167D312 : {33C1836C4FBB410A988935BB2DAFEA35={InputName=33C1836C4FBB410A988935BB2DAFEA35}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:41,684 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 33C1836C4FBB410A988935BB2DAFEA35, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:41,684 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,684 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463521961509_0001, dagName=[7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter
2016-05-17 21:52:41,685 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,685 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,685 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]. Starting root input initializers: 1
2016-05-17 21:52:41,685 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:41,686 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 33C1836C4FBB410A988935BB2DAFEA35 on vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,687 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:41,687 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 33C1836C4FBB410A988935BB2DAFEA35 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:41,687 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:41,688 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:41,689 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:41,689 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 33C1836C4FBB410A988935BB2DAFEA35 on vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,690 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:41,690 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {33C1836C4FBB410A988935BB2DAFEA35=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:41,690 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] parallelism set to 1
2016-05-17 21:52:41,690 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:41,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521961509_0001_1_00, eventCount=1
2016-05-17 21:52:41,691 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,691 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=817EF10BD9EE44FE91419FE44ACE46C5, vertexId=vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:41,691 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=817EF10BD9EE44FE91419FE44ACE46C5, vertexId=vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,693 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=817EF10BD9EE44FE91419FE44ACE46C5, vertexName=CDED5065C842484FB462A32D3167D312, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:41,693 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=817EF10BD9EE44FE91419FE44ACE46C5, vertexId=vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CDED5065C842484FB462A32D3167D312, vertexId=vertex_1463521961509_0001_1_00, initRequestedTime=1463521961684, initedTime=1463521961694, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:41,694 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] with distanceFromRoot: 0
2016-05-17 21:52:41,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:41,695 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:41,695 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CDED5065C842484FB462A32D3167D312
2016-05-17 21:52:41,695 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,696 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:41,696 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521961509_0001_1_00, startRequestedTime=1463521961695, startedTime=1463521961695
2016-05-17 21:52:41,696 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:41,697 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:41,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:TASK_STARTED]: vertexName=CDED5065C842484FB462A32D3167D312, taskId=task_1463521961509_0001_1_00_000000, scheduledTime=1463521961696, launchTime=1463521961696
2016-05-17 21:52:41,697 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961509_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:41,697 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,698 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter, VertexName: CDED5065C842484FB462A32D3167D312, VertexParallelism: 1, TaskAttemptID:attempt_1463521961509_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=33C1836C4FBB410A988935BB2DAFEA35, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=817EF10BD9EE44FE91419FE44ACE46C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:41,698 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961509_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:41,698 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:41,699 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:41,699 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:41,699 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:41,700 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521961509_0001_00_000001
2016-05-17 21:52:41,700 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521961509_0001, with dag id: dag_1463521961509_0001_1
2016-05-17 21:52:41,700 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:41,702 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521961509_0001_00_000001
2016-05-17 21:52:41,702 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:41,703 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:41,703 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:41,703 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521961509_0001_00_000001, launchTime=1463521961703
2016-05-17 21:52:41,704 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961509_0001_00_000001
2016-05-17 21:52:41,706 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521961509_0001_1_00_000000_0] to container: [container_1463521961509_0001_00_000001]
2016-05-17 21:52:41,706 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:41,706 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521961509_0001_1_00_000000_0] started. Is using containerId: [container_1463521961509_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:41,706 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CDED5065C842484FB462A32D3167D312, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0, startTime=1463521961706, containerId=container_1463521961509_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521961509_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:41,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:41,707 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521961509_0001_00_000001 given task: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,707 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961509_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:41,707 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961509_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:41,707 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961509_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41 Starting to run new task attempt: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,708 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,708 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,708 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:41,708 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:41,708 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:41,708 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:41,708 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:41,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8] test/(1/1) ...latform/testFilter/filter, VertexName: CDED5065C842484FB462A32D3167D312, VertexParallelism: 1, TaskAttemptID:attempt_1463521961509_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=33C1836C4FBB410A988935BB2DAFEA35, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=817EF10BD9EE44FE91419FE44ACE46C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:41,709 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:41,709 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:41,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,710 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521961509_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:41,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=33C1836C4FBB410A988935BB2DAFEA35, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:41,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=817EF10BD9EE44FE91419FE44ACE46C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:41,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:41,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:41,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,712 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 817EF10BD9EE44FE91419FE44ACE46C5
2016-05-17 21:52:41,712 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:41,712 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 817EF10BD9EE44FE91419FE44ACE46C5
2016-05-17 21:52:41,713 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:41,714 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:41,714 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,721 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:41,721 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 817EF10BD9EE44FE91419FE44ACE46C5
2016-05-17 21:52:41,722 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CDED5065C842484FB462A32D3167D312, ordinal: 0
2016-05-17 21:52:41,722 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDED5065C842484FB462A32D3167D312, mem on start (mb), free: 341, total: 555, max: 672
2016-05-17 21:52:41,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:41,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:41,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:41,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:41,723 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:41,723 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:41,723 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:41,723 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:41,723 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:41,724 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:41,724 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:52:41,724 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 33C1836C4FBB410A988935BB2DAFEA35, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:41,724 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 817EF10BD9EE44FE91419FE44ACE46C5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:41,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:41,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 33C1836C4FBB410A988935BB2DAFEA35 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:41,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,725 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:41,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:41,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:41,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:41,725 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,725 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:41,726 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@250eaa67
2016-05-17 21:52:41,727 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@250eaa67
2016-05-17 21:52:41,727 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:41,727 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:41,730 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:41,730 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 33C1836C4FBB410A988935BB2DAFEA35
2016-05-17 21:52:41,730 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"], id: 817EF10BD9EE44FE91419FE44ACE46C5
2016-05-17 21:52:41,731 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-17 21:52:41,731 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:41,731 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:41,731 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CDED5065C842484FB462A32D3167D312, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:41,733 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:41,733 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:41,734 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,734 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521961509_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:52:41,735 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,735 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDED5065C842484FB462A32D3167D312, mem on close (mb), free: 339, total: 555, max: 672
2016-05-17 21:52:41 Completed running task attempt: attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,735 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:41,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CDED5065C842484FB462A32D3167D312, taskAttemptId=attempt_1463521961509_0001_1_00_000000_0, startTime=1463521961706, finishTime=1463521961736, timeTaken=30, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 21:52:41,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:41,737 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961509_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:41,737 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:41,737 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521961509_0001_1_00_000000_0
2016-05-17 21:52:41,738 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:41,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:TASK_FINISHED]: vertexName=CDED5065C842484FB462A32D3167D312, taskId=task_1463521961509_0001_1_00_000000, startTime=1463521961706, finishTime=1463521961737, timeTaken=31, status=SUCCEEDED, successfulAttemptID=attempt_1463521961509_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 21:52:41,738 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961509_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:41,738 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] : 1
2016-05-17 21:52:41,738 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=3]]
2016-05-17 21:52:41,739 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:41,739 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,740 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961509_0001_00_000001
2016-05-17 21:52:41,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:VERTEX_FINISHED]: vertexName=CDED5065C842484FB462A32D3167D312, vertexId=vertex_1463521961509_0001_1_00, initRequestedTime=1463521961684, initedTime=1463521961694, startRequestedTime=1463521961695, startedTime=1463521961695, finishTime=1463521961738, timeTaken=43, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3, vertexStats=firstTaskStartTime=1463521961706, firstTasksToStart=[ task_1463521961509_0001_1_00_000000 ], lastTaskFinishTime=1463521961736, lastTasksToFinish=[ task_1463521961509_0001_1_00_000000 ], minTaskDuration=30, maxTaskDuration=30, avgTaskDuration=30.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521961509_0001_1_00_000000 ], longestDurationTasks=[ task_1463521961509_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:41,741 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:41,741 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:41,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521961509_0001_00_000001, stoppedTime=1463521961741, exitStatus=0
2016-05-17 21:52:41,741 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:41,741 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:52:41,741 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:41,741 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521961509_0001_1
2016-05-17 21:52:41,742 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521961509_0001_1
2016-05-17 21:52:41,742 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 817EF10BD9EE44FE91419FE44ACE46C5 for vertex: vertex_1463521961509_0001_1_00 [CDED5065C842484FB462A32D3167D312]
2016-05-17 21:52:41,742 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:41,750 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521961509_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:41,750 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961509_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:41,750 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521961509_0001_00_000001, Exiting
2016-05-17 21:52:41,750 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521961509_0001_00_000001
2016-05-17 21:52:41,750 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521961509_0001_00_000001
2016-05-17 21:52:41,752 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521961509_0001_1, queueSize=0
2016-05-17 21:52:41,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:41,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961509_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521961509_0001_1, startTime=1463521961683, finishTime=1463521961751, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=582483968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463521961731, Process_Duration=4, Process_End_Time=1463521961735, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 21:52:41,752 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521961509_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:41,752 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961509_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:41 Completed Dag: dag_1463521961509_0001_1
2016-05-17 21:52:41,753 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521961509_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:41,753 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521961509_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:41,753 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521961509_0001_1_post
2016-05-17 21:52:41,753 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:41,753 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:41,753 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:41,756 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521961509_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:41,756 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521961509_0001_00_000001 completed successfully
2016-05-17 21:52:41,757 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961509_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:41,759 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463521961509_0001
2016-05-17 21:52:41,761 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:41,761 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:41,761 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:41,761 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:41,762 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:41,762 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:41,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:41,762 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:41,763 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:41,763 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:41,763 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:41,763 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:41,763 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:41,768 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7D0EB52BD5314FBAA696ECF436EE1C7C/E5F6B12C5F9C40DD8A212940AE7318B8/.tez/application_1463521961509_0001
2016-05-17 21:52:41,768 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter/_temporary
2016-05-17 21:52:41,769 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.276
2016-05-17 21:52:41,810 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-17 21:52:41,836 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:52:41,837 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:52:41,875 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:52:41,889 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-17 21:52:41,889 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:41,890 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:52:41,890 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:52:41,905 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 21:52:41,912 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple
2016-05-17 21:52:41,973 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 21:52:41,973 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:41,973 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-17 21:52:41,974 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 21:52:41,974 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 21:52:41,974 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 21:52:41,975 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testSimple/simple
2016-05-17 21:52:41,975 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:52:41,987 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 21:52:41,991 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:41,992 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:52:41,992 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:52:42,001 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001 doesn't exist and is created
2016-05-17 21:52:42,016 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:52:42,016 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:52:42,017 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001
2016-05-17 21:52:42,020 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463521961992_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:52:42,031 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:52:42,032 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:52:42,034 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:52:42,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:52:42,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001/localmode-log-dir
2016-05-17 21:52:42,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001/localmode-log-dir/history.txt.appattempt_1463521961992_0001_000000, maxErrors=10
2016-05-17 21:52:42,035 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:52:42,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463521961992_0001_000000, appSubmitTime=1463521962019, launchTime=1463521962020
2016-05-17 21:52:42,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:52:42,042 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:52:42,043 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:52:42,043 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:52:42,044 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43199
2016-05-17 21:52:42,046 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:52:42,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463521961992_0001_000000, startTime=1463521962048
2016-05-17 21:52:42,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:52:42,048 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:52:42,117 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:52:42,117 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:52:42,117 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463521961992_0001, dagName=[8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple
2016-05-17 21:52:42,131 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:52:42,131 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:52:42,132 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple
2016-05-17 21:52:42,134 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463521961992_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001/localmode-log-dir/dag_1463521961992_0001_1.dot
2016-05-17 21:52:42,134 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001/localmode-log-dir/dag_1463521961992_0001_1-tez-dag.pb.txt
2016-05-17 21:52:42,147 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521961992_0001_1
2016-05-17 21:52:42,147 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,147 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple
2016-05-17 21:52:42 Running Dag: dag_1463521961992_0001_1
2016-05-17 21:52:42,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463521961992_0001_1, submitTime=1463521962132
2016-05-17 21:52:42,163 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:52:42,164 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,164 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:52:42,165 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:52:42,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463521961992_0001_1, initTime=1463521962163
2016-05-17 21:52:42,165 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961992_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:52:42,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_STARTED]: dagID=dag_1463521961992_0001_1, startTime=1463521962166
2016-05-17 21:52:42,166 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:52:42,166 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:52:42,166 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961992_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:52:42,166 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ED43F9BE1D8A4519B18FE2FDF9494E3D : {4E43EBC6419B4C5EAC4D309A32FA9BCA={InputName=4E43EBC6419B4C5EAC4D309A32FA9BCA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:52:42,167 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4E43EBC6419B4C5EAC4D309A32FA9BCA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:52:42,167 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,167 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463521961992_0001, dagName=[8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple
2016-05-17 21:52:42,167 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,167 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,167 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]. Starting root input initializers: 1
2016-05-17 21:52:42,168 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:52:42,168 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,168 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4E43EBC6419B4C5EAC4D309A32FA9BCA on vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,170 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:52:42,170 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4E43EBC6419B4C5EAC4D309A32FA9BCA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:52:42,171 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:52:42,171 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:52:42,171 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:52:42,171 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,172 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,172 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,172 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2194406ED73040119A986EB3879DF0C1, vertexId=vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:52:42,173 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 21:52:42,173 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2194406ED73040119A986EB3879DF0C1, vertexId=vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,175 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2194406ED73040119A986EB3879DF0C1, vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:52:42,175 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2194406ED73040119A986EB3879DF0C1, vertexId=vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,173 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4E43EBC6419B4C5EAC4D309A32FA9BCA on vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,177 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:42,177 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, vertexId=vertex_1463521961992_0001_1_01, initRequestedTime=1463521962168, initedTime=1463521962176, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:52:42,177 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:52:42,177 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:42,177 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4E43EBC6419B4C5EAC4D309A32FA9BCA=forAllWorkUnits=true, update=[1]}
2016-05-17 21:52:42,178 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] parallelism set to 1
2016-05-17 21:52:42,178 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:52:42,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521961992_0001_1_00, eventCount=1
2016-05-17 21:52:42,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:52:42,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, vertexId=vertex_1463521961992_0001_1_00, initRequestedTime=1463521962166, initedTime=1463521962179, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:52:42,180 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] with distanceFromRoot: 0
2016-05-17 21:52:42,180 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:52:42,180 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,180 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,181 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:42,181 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521961992_0001_1_00, startRequestedTime=1463521962180, startedTime=1463521962180
2016-05-17 21:52:42,181 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:42,181 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:42,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_STARTED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, taskId=task_1463521961992_0001_1_00_000000, scheduledTime=1463521962181, launchTime=1463521962181
2016-05-17 21:52:42,182 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:42,182 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463521961992_0001_1_00 for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] numStartedSources: 1 numSources: 1
2016-05-17 21:52:42,182 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] with distanceFromRoot: 1
2016-05-17 21:52:42,182 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,182 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ED43F9BE1D8A4519B18FE2FDF9494E3D in vertex: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,183 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A6D55EBC661049FEA8EDA1BB694B72E1 with 1 source tasks and 1 pending tasks
2016-05-17 21:52:42,183 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:52:42,183 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463521961992_0001_1_01, startRequestedTime=1463521962182, startedTime=1463521962182
2016-05-17 21:52:42,183 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:52:42,184 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple, VertexName: ED43F9BE1D8A4519B18FE2FDF9494E3D, VertexParallelism: 1, TaskAttemptID:attempt_1463521961992_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4E43EBC6419B4C5EAC4D309A32FA9BCA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A6D55EBC661049FEA8EDA1BB694B72E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:42,184 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:42,184 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:42,184 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463521961992_0001, with dag id: dag_1463521961992_0001_1
2016-05-17 21:52:42,185 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:52:42,185 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:52:42,185 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:52:42,186 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521961992_0001_00_000001
2016-05-17 21:52:42,186 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:42,188 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521961992_0001_00_000001
2016-05-17 21:52:42,188 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:42,189 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:42,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:42,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521961992_0001_00_000001, launchTime=1463521962190
2016-05-17 21:52:42,191 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961992_0001_00_000001
2016-05-17 21:52:42,193 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521961992_0001_1_00_000000_0] to container: [container_1463521961992_0001_00_000001]
2016-05-17 21:52:42,193 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:42,194 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521961992_0001_00_000001 given task: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,194 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961992_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42 Starting to run new task attempt: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,194 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521961992_0001_1_00_000000_0] started. Is using containerId: [container_1463521961992_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:52:42,194 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,195 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,195 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:42,195 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0, startTime=1463521962194, containerId=container_1463521961992_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521961992_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:52:42,195 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:42,196 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:42,196 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:42,196 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:42,196 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:42,196 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:42,196 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:42,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple, VertexName: ED43F9BE1D8A4519B18FE2FDF9494E3D, VertexParallelism: 1, TaskAttemptID:attempt_1463521961992_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4E43EBC6419B4C5EAC4D309A32FA9BCA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A6D55EBC661049FEA8EDA1BB694B72E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:52:42,197 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:42,197 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:42,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,198 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521961992_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:52:42,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4E43EBC6419B4C5EAC4D309A32FA9BCA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:52:42,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A6D55EBC661049FEA8EDA1BB694B72E1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:52:42,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,201 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:42,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:42,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,203 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:42,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,208 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ED43F9BE1D8A4519B18FE2FDF9494E3D, ordinal: 0
2016-05-17 21:52:42,208 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ED43F9BE1D8A4519B18FE2FDF9494E3D, mem on start (mb), free: 474, total: 555, max: 672
2016-05-17 21:52:42,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:42,208 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:52:42,209 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:42,210 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:52:42,211 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:42,211 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:42,212 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:52:42,212 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:42,212 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:52:42,212 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:42,213 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:52:42,213 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4E43EBC6419B4C5EAC4D309A32FA9BCA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:52:42,213 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A6D55EBC661049FEA8EDA1BB694B72E1, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:52:42,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:42,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4E43EBC6419B4C5EAC4D309A32FA9BCA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:42,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:42,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:42,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,214 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:42,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:42,215 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,215 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:52:42,216 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5958b0ae
2016-05-17 21:52:42,217 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5958b0ae
2016-05-17 21:52:42,217 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:52:42,217 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:52:42,220 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:42,220 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 4E43EBC6419B4C5EAC4D309A32FA9BCA
2016-05-17 21:52:42,221 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: D334DFA40CE8414CBD0B9448AEA4B472
2016-05-17 21:52:42,221 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] D334DFA40CE8414CBD0B9448AEA4B472
2016-05-17 21:52:42,222 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@128be4e8; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1e6144db; comparator=cascading.tuple.hadoop.util.TupleComparator@3ef9572; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:52:42,222 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:52:42,223 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:52:42,223 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:42,236 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:52:42,237 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:52:42,237 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:52:42,237 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:52:42,237 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:52:42,238 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 21:52:42,238 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:52:42,238 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ED43F9BE1D8A4519B18FE2FDF9494E3D, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:42,241 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,241 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ED43F9BE1D8A4519B18FE2FDF9494E3D, mem on close (mb), free: 371, total: 555, max: 672
2016-05-17 21:52:42,241 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:52:42,241 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:52:42,241 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 21:52:42,241 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 21:52:42,245 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:52:42 Completed running task attempt: attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,248 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:42,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463521962239, Process_Duration=2, Process_End_Time=1463521962241, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 21:52:42,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463521961992_0001_1_00, eventCount=1
2016-05-17 21:52:42,249 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:42,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, taskAttemptId=attempt_1463521961992_0001_1_00_000000_0, startTime=1463521962194, finishTime=1463521962249, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962239, Process_Duration=2, Process_End_Time=1463521962241, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:42,250 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:42,250 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521961992_0001_1_00_000000_0
2016-05-17 21:52:42,250 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:42,250 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:42,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_FINISHED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, taskId=task_1463521961992_0001_1_00_000000, startTime=1463521962194, finishTime=1463521962250, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463521961992_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962239, Process_Duration=2, Process_End_Time=1463521962241, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 21:52:42,251 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:42,251 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] : 1
2016-05-17 21:52:42,251 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,251 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:42,251 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,252 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961992_0001_00_000001
2016-05-17 21:52:42,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:42,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_FINISHED]: vertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, vertexId=vertex_1463521961992_0001_1_00, initRequestedTime=1463521962166, initedTime=1463521962179, startRequestedTime=1463521962180, startedTime=1463521962180, finishTime=1463521962251, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962239, Process_Duration=2, Process_End_Time=1463521962241, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463521962194, firstTasksToStart=[ task_1463521961992_0001_1_00_000000 ], lastTaskFinishTime=1463521962249, lastTasksToFinish=[ task_1463521961992_0001_1_00_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521961992_0001_1_00_000000 ], longestDurationTasks=[ task_1463521961992_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:42,252 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:42,252 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:42,253 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] attempt: attempt_1463521961992_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:52:42,253 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A6D55EBC661049FEA8EDA1BB694B72E1
2016-05-17 21:52:42,253 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521961992_0001_00_000001, stoppedTime=1463521962253, exitStatus=0
2016-05-17 21:52:42,253 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:42,253 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:42,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:42,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:52:42,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_STARTED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, taskId=task_1463521961992_0001_1_01_000000, scheduledTime=1463521962253, launchTime=1463521962253
2016-05-17 21:52:42,254 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:52:42,255 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,255 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463521961992_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:52:42,255 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple, VertexName: A6D55EBC661049FEA8EDA1BB694B72E1, VertexParallelism: 1, TaskAttemptID:attempt_1463521961992_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2194406ED73040119A986EB3879DF0C1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:42,255 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:52:42,255 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:52:42,255 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:52:42,257 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463521961992_0001_00_000002
2016-05-17 21:52:42,257 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:52:42,259 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463521961992_0001_00_000002
2016-05-17 21:52:42,259 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:52:42,260 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:52:42,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:52:42,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463521961992_0001_00_000002, launchTime=1463521962259
2016-05-17 21:52:42,260 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961992_0001_00_000002
2016-05-17 21:52:42,261 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463521961992_0001_1_01_000000_0] to container: [container_1463521961992_0001_00_000002]
2016-05-17 21:52:42,261 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:52:42,262 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463521961992_0001_1_01_000000_0] started. Is using containerId: [container_1463521961992_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:52:42,262 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0, startTime=1463521962262, containerId=container_1463521961992_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463521961992_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:52:42,262 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:52:42,262 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521961992_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:52:42,262 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961992_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:42,262 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:52:42,263 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521961992_0001_00_000001, Exiting
2016-05-17 21:52:42,263 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:52:42,263 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463521961992_0001_00_000002 given task: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,263 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521961992_0001_00_000001
2016-05-17 21:52:42,263 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521961992_0001_00_000001
2016-05-17 21:52:42,264 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961992_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42 Starting to run new task attempt: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,264 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,264 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,265 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:52:42,265 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:52:42,265 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:52:42,265 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:52:42,265 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:52:42,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE] test/(1/1) ...latform/testSimple/simple, VertexName: A6D55EBC661049FEA8EDA1BB694B72E1, VertexParallelism: 1, TaskAttemptID:attempt_1463521961992_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2194406ED73040119A986EB3879DF0C1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:52:42,265 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521961992_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:42,265 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:52:42,265 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521961992_0001_00_000001 completed successfully
2016-05-17 21:52:42,266 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:52:42,266 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:52:42,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,267 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463521961992_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:52:42,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ED43F9BE1D8A4519B18FE2FDF9494E3D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:52:42,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:52:42,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2194406ED73040119A986EB3879DF0C1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:52:42,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,270 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:52:42,270 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:52:42,270 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,271 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:52:42,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:52:42,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2194406ED73040119A986EB3879DF0C1
2016-05-17 21:52:42,273 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:52:42,274 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2194406ED73040119A986EB3879DF0C1
2016-05-17 21:52:42,276 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A6D55EBC661049FEA8EDA1BB694B72E1, ordinal: 1
2016-05-17 21:52:42,276 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6D55EBC661049FEA8EDA1BB694B72E1, mem on start (mb), free: 367, total: 555, max: 672
2016-05-17 21:52:42,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:52:42,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:52:42,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:52:42,283 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:52:42,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2194406ED73040119A986EB3879DF0C1
2016-05-17 21:52:42,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:52:42,283 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:52:42,283 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:52:42,283 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:52:42,284 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:52:42,284 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:52:42,284 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:52:42,284 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ED43F9BE1D8A4519B18FE2FDF9494E3D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:52:42,284 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2194406ED73040119A986EB3879DF0C1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:52:42,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:52:42,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ED43F9BE1D8A4519B18FE2FDF9494E3D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:52:42,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:52:42,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:52:42,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,287 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:52:42,288 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:52:42,288 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ED43F9BE1D8A4519B18FE2FDF9494E3D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:52:42,289 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:52:42,289 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:52:42,289 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:52:42,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:52:42,292 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,293 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463521961992_0001_1_00_000000_0_10089, runDuration: 39458, ]
2016-05-17 21:52:42,295 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463521961992_0001_1_00_000000_0_10089] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463521962295, TimeTaken=2, Rate=0.10 MB/s
2016-05-17 21:52:42,295 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,295 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:52:42,295 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [ED43F9BE1D8A4519B18FE2FDF9494E3D] #14 in 2ms
2016-05-17 21:52:42,296 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ED43F9BE1D8A4519B18FE2FDF9494E3D] #14, status:false, isInterrupted:false
2016-05-17 21:52:42,296 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 21:52:42,297 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:52:42,298 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: D334DFA40CE8414CBD0B9448AEA4B472
2016-05-17 21:52:42,298 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"], id: 2194406ED73040119A986EB3879DF0C1
2016-05-17 21:52:42,298 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 21:52:42,298 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:42,298 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:52:42,299 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 21:52:42,299 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,299 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:52:42,298 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-17 21:52:42,300 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] D334DFA40CE8414CBD0B9448AEA4B472, for 1 inputs
2016-05-17 21:52:42,300 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A6D55EBC661049FEA8EDA1BB694B72E1, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:52:42,302 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@9d68d7a; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@64e249a1
2016-05-17 21:52:42,305 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:52:42,306 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:52:42,307 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,307 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463521961992_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:52:42,308 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,308 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6D55EBC661049FEA8EDA1BB694B72E1, mem on close (mb), free: 363, total: 555, max: 672
2016-05-17 21:52:42,308 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ED43F9BE1D8A4519B18FE2FDF9494E3D
2016-05-17 21:52:42,309 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:52:42,309 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:52:42,310 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:52:42,310 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:52:42,310 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:52:42 Completed running task attempt: attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,312 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:52:42,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463521962300, Process_Duration=8, Process_End_Time=1463521962308, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 21:52:42,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, taskAttemptId=attempt_1463521961992_0001_1_01_000000_0, startTime=1463521962262, finishTime=1463521962313, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962300, Process_Duration=8, Process_End_Time=1463521962308, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:42,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:52:42,314 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:52:42,314 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,313 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463521961992_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:52:42,314 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463521961992_0001_00_000002
2016-05-17 21:52:42,314 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463521961992_0001_1_01_000000_0
2016-05-17 21:52:42,315 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:52:42,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:TASK_FINISHED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, taskId=task_1463521961992_0001_1_01_000000, startTime=1463521962262, finishTime=1463521962315, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463521961992_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962300, Process_Duration=8, Process_End_Time=1463521962308, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 21:52:42,315 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463521961992_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:52:42,315 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] : 1
2016-05-17 21:52:42,316 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,316 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:52:42,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:52:42,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:VERTEX_FINISHED]: vertexName=A6D55EBC661049FEA8EDA1BB694B72E1, vertexId=vertex_1463521961992_0001_1_01, initRequestedTime=1463521962168, initedTime=1463521962176, startRequestedTime=1463521962182, startedTime=1463521962182, finishTime=1463521962316, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=581959680, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463521962300, Process_Duration=8, Process_End_Time=1463521962308, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463521962262, firstTasksToStart=[ task_1463521961992_0001_1_01_000000 ], lastTaskFinishTime=1463521962313, lastTasksToFinish=[ task_1463521961992_0001_1_01_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463521961992_0001_1_01_000000 ], longestDurationTasks=[ task_1463521961992_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:52:42,317 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:52:42,318 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:52:42,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463521961992_0001_00_000002, stoppedTime=1463521962318, exitStatus=0
2016-05-17 21:52:42,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:52:42,318 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:52:42,319 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:52:42,319 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463521961992_0001_1
2016-05-17 21:52:42,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463521961992_0001_1
2016-05-17 21:52:42,320 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2194406ED73040119A986EB3879DF0C1 for vertex: vertex_1463521961992_0001_1_01 [A6D55EBC661049FEA8EDA1BB694B72E1]
2016-05-17 21:52:42,325 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463521961992_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:52:42,325 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463521961992_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:52:42,326 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463521961992_0001_00_000002, Exiting
2016-05-17 21:52:42,326 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463521961992_0001_00_000002
2016-05-17 21:52:42,326 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463521961992_0001_00_000002
2016-05-17 21:52:42,326 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463521961992_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:52:42,326 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463521961992_0001_00_000002 completed successfully
2016-05-17 21:52:42,327 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463521961992_0001_1_00 [ED43F9BE1D8A4519B18FE2FDF9494E3D]
2016-05-17 21:52:42,329 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463521961992_0001_1, queueSize=0
2016-05-17 21:52:42,329 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:52:42,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463521961992_0001_1][Event:DAG_FINISHED]: dagId=dag_1463521961992_0001_1, startTime=1463521962166, finishTime=1463521962327, timeTaken=161, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1163919360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927043924539, Process_Duration=10, Process_End_Time=2927043924549, Read_Duration=0, Tuples_Read=20, Tuples_Written=18, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 21:52:42,329 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463521961992_0001_1 finished with state: SUCCEEDED
2016-05-17 21:52:42,329 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463521961992_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:52:42,330 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463521961992_0001_00_000002
2016-05-17 21:52:42,330 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463521961992_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:52:42 Completed Dag: dag_1463521961992_0001_1
2016-05-17 21:52:42,333 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463521961992_0001_1, dagState=SUCCEEDED
2016-05-17 21:52:42,334 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463521961992_0001_1_post
2016-05-17 21:52:42,334 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:52:42,334 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:52:42,336 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463521961992_0001
2016-05-17 21:52:42,338 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:52:42,338 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:52:42,338 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:52:42,338 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:52:42,338 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:52:42,338 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:52:42,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:52:42,340 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:52:42,340 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:52:42,340 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:52:42,340 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:52:42,341 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:52:42,341 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:52:42,346 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8039E28ACC044922BACB09F494E9103C/AEC5CFE5275E4EDF9CC72F70341734FE/.tez/application_1463521961992_0001
2016-05-17 21:52:42,346 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple/_temporary
2016-05-17 21:52:42,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.373
2016-05-17 21:52:42,382 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimple/simple