Class cascading.BasicPipesPlatformTest

16

tests

0

failures

0

ignored

15.298s

duration

100%

successful

Tests

Test Duration Result
testCoGroup 1.213s passed
testCoGroupRelativeUnknown 1.191s passed
testCount 1.011s passed
testCount2 0.637s passed
testCount3 0.633s passed
testCount4 0.586s passed
testFilter 0.391s passed
testFilterAll 0.424s passed
testReplace 0.432s passed
testSimple 0.582s passed
testSimpleChain 4.390s passed
testSimpleRelative 0.946s passed
testSimpleRelativeUnknown 0.576s passed
testSimpleResult 1.194s passed
testSwap 0.566s passed
testUnGroup 0.526s passed

Standard output

2016-05-17 20:45:42,641 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-17 20:45:42,663 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.BasicPipesPlatformTest, with platform: hadoop2-tez
2016-05-17 20:45:42,688 INFO  tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-17 20:45:42,999 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-17 20:45:43,341 INFO  platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides: 
2016-05-17 20:45:43,568 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:43,570 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:43,695 INFO  property.AppProps (AppProps.java:getAppID(177)) - using app.id: 0FD517CFF093459A99073014D46E7D06
2016-05-17 20:45:43,898 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.091
2016-05-17 20:45:43,935 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.129
2016-05-17 20:45:43,937 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:43,937 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:43,939 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:44,086 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:44,261 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-17 20:45:44,490 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-17 20:45:44,493 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:44,494 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:44,495 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-17 20:45:44,495 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:44,495 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:44,497 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:44,498 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:44,499 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:44,516 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:44,668 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:44,674 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:44,674 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:44,715 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001 doesn't exist and is created
2016-05-17 20:45:44,773 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:44,773 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:44,775 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001
2016-05-17 20:45:44,806 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517944674_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:44,846 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:44,862 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:44,875 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-17 20:45:44,875 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-17 20:45:44,875 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:44,888 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:44,892 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:44,895 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001/localmode-log-dir
2016-05-17 20:45:44,896 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001/localmode-log-dir/history.txt.appattempt_1463517944674_0001_000000, maxErrors=10
2016-05-17 20:45:44,896 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:44,899 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517944674_0001_000000, appSubmitTime=1463517944792, launchTime=1463517944805
2016-05-17 20:45:44,909 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:44,909 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:44,914 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:44,916 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:44,976 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 20:45:44,976 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 20:45:44,976 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:44,980 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41451
2016-05-17 20:45:45,022 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:45,077 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 20:45:45,077 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 20:45:45,077 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:45,090 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:45,091 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517944674_0001_000000, startTime=1463517945090
2016-05-17 20:45:45,091 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:45,177 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:45,194 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:45,195 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463517944674_0001, dagName=[52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:45,418 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:45,525 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517944674_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001/localmode-log-dir/dag_1463517944674_0001_1.dot
2016-05-17 20:45:45,530 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001/localmode-log-dir/dag_1463517944674_0001_1-tez-dag.pb.txt
2016-05-17 20:45:45,627 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517944674_0001_1
2016-05-17 20:45:45,627 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:45,627 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:45 Running Dag: dag_1463517944674_0001_1
2016-05-17 20:45:45,672 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:45,672 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517944674_0001_1, submitTime=1463517945418
2016-05-17 20:45:45,706 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:45,715 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:45,717 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517944674_0001_1, initTime=1463517945674
2016-05-17 20:45:45,718 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517944674_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:45,718 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:45,720 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_STARTED]: dagID=dag_1463517944674_0001_1, startTime=1463517945719
2016-05-17 20:45:45,720 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:45,721 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:45,722 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517944674_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:45,723 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B7CF60ABA0644A7FA70974D7766572AB : {7EDF6B7FD6664EBA8177ECC7A9D7A4EB={InputName=7EDF6B7FD6664EBA8177ECC7A9D7A4EB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:45,723 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:45,723 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,730 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,730 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,733 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]. Starting root input initializers: 1
2016-05-17 20:45:45,736 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463517944674_0001, dagName=[52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:45,773 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB on vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,792 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:45,795 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:45,797 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7EDF6B7FD6664EBA8177ECC7A9D7A4EB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:45,799 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:45,799 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:45,802 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:45,831 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:45,831 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:45,845 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:45,848 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB on vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,849 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:45,852 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:45,852 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, vertexId=vertex_1463517944674_0001_1_01, initRequestedTime=1463517945792, initedTime=1463517945851, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:45,853 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:45,853 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:45,853 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7EDF6B7FD6664EBA8177ECC7A9D7A4EB=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:45,854 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] parallelism set to 1
2016-05-17 20:45:45,854 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:45,860 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517944674_0001, with dag id: dag_1463517944674_0001_1
2016-05-17 20:45:45,869 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517944674_0001_1_00, eventCount=1
2016-05-17 20:45:45,870 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:45,870 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, vertexId=vertex_1463517944674_0001_1_00, initRequestedTime=1463517945723, initedTime=1463517945870, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:45,871 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] with distanceFromRoot: 0
2016-05-17 20:45:45,872 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:45,876 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,894 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:45,894 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,895 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,895 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,896 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A7B573490F9540059E3AEE82DB4B0EF9, vertexId=vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:45,902 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A7B573490F9540059E3AEE82DB4B0EF9, vertexId=vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,920 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A7B573490F9540059E3AEE82DB4B0EF9, vertexName=DA216359D1C94EB2B0C8F756359FEFD6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:45,938 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A7B573490F9540059E3AEE82DB4B0EF9, vertexId=vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,942 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:45,942 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, vertexId=vertex_1463517944674_0001_1_02, initRequestedTime=1463517945876, initedTime=1463517945942, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:45,943 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:45,944 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:45,949 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,951 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:45,951 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517944674_0001_1_00, startRequestedTime=1463517945944, startedTime=1463517945944
2016-05-17 20:45:45,953 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:45,981 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:45,981 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_STARTED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, taskId=task_1463517944674_0001_1_00_000000, scheduledTime=1463517945981, launchTime=1463517945981
2016-05-17 20:45:45,982 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:45,982 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517944674_0001_1_00 for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] numStartedSources: 1 numSources: 1
2016-05-17 20:45:45,983 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] with distanceFromRoot: 1
2016-05-17 20:45:45,985 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,989 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B7CF60ABA0644A7FA70974D7766572AB in vertex: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:45,992 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B99DE4A7E23B4D85A8EE364452481C90 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:45,992 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:45,992 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517944674_0001_1_01, startRequestedTime=1463517945986, startedTime=1463517945986
2016-05-17 20:45:45,993 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:45,995 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: B7CF60ABA0644A7FA70974D7766572AB, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7EDF6B7FD6664EBA8177ECC7A9D7A4EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:45,997 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:45,997 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517944674_0001_1_01 for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] numStartedSources: 1 numSources: 1
2016-05-17 20:45:45,998 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] with distanceFromRoot: 2
2016-05-17 20:45:45,998 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:45,998 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B99DE4A7E23B4D85A8EE364452481C90 in vertex: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:45,999 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DA216359D1C94EB2B0C8F756359FEFD6 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:45,999 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:46,000 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517944674_0001_1_02, startRequestedTime=1463517945998, startedTime=1463517945998
2016-05-17 20:45:46,000 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:46,038 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:46,042 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:46,042 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:46,046 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517944674_0001_00_000001
2016-05-17 20:45:46,046 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:46,052 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517944674_0001_00_000001
2016-05-17 20:45:46,052 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:46,057 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:46,058 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:46,059 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517944674_0001_00_000001, launchTime=1463517946058
2016-05-17 20:45:46,062 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000001
2016-05-17 20:45:46,068 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517944674_0001_1_00_000000_0] to container: [container_1463517944674_0001_00_000001]
2016-05-17 20:45:46,068 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:46,071 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517944674_0001_1_00_000000_0] started. Is using containerId: [container_1463517944674_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:46,071 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517944674_0001_00_000001 given task: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,072 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,072 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:46,072 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0, startTime=1463517946069, containerId=container_1463517944674_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517944674_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:46,077 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:46,081 INFO  counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters:  GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-17 20:45:46 Starting to run new task attempt: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,082 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,082 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,083 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:46,084 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:46,084 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:46,086 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:46,089 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:46,121 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:46,122 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: B7CF60ABA0644A7FA70974D7766572AB, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7EDF6B7FD6664EBA8177ECC7A9D7A4EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:46,124 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:46,129 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:46,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,131 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517944674_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:46,137 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7EDF6B7FD6664EBA8177ECC7A9D7A4EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:46,138 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,139 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:46,141 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:46,143 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:46,144 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,153 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,167 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:46,168 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,171 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:46,173 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:46,174 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,175 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,186 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B7CF60ABA0644A7FA70974D7766572AB, ordinal: 0
2016-05-17 20:45:46,186 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B7CF60ABA0644A7FA70974D7766572AB, mem on start (mb), free: 228, total: 281, max: 672
2016-05-17 20:45:46,186 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:46,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:46,188 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:46,190 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:46,195 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:46,196 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:46,196 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:46,197 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:46,198 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:46,198 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7EDF6B7FD6664EBA8177ECC7A9D7A4EB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:46,198 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B99DE4A7E23B4D85A8EE364452481C90, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:46,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:46,201 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:46,201 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:46,203 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:46,201 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,204 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:46,205 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,205 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:46,209 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,210 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:46,220 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@75dafc0b
2016-05-17 20:45:46,233 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@75dafc0b
2016-05-17 20:45:46,236 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:46,236 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:46,248 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:46,278 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,278 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: F5ADF73F391F48779399EEE33FA9E9C0
2016-05-17 20:45:46,279 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] F5ADF73F391F48779399EEE33FA9E9C0
2016-05-17 20:45:46,324 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3676fc91; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3dbfaf86; comparator=cascading.tuple.hadoop.util.TupleComparator@8dc2faa; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:46,329 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:46,332 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:46,332 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:46,445 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:46,445 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:46,445 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:46,446 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:46,446 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:46,449 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:46,449 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:46,451 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B7CF60ABA0644A7FA70974D7766572AB, all 1 inputs ready in: 00:00:00.001
2016-05-17 20:45:46,459 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,459 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B7CF60ABA0644A7FA70974D7766572AB, mem on close (mb), free: 116, total: 281, max: 672
2016-05-17 20:45:46,460 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:46,460 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:46,460 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 20:45:46,460 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 20:45:46,478 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:46 Completed running task attempt: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,498 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:46,499 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517944674_0001_1_00, eventCount=1
2016-05-17 20:45:46,503 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517946451, Process_Duration=8, Process_End_Time=1463517946459, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 20:45:46,503 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:46,504 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:46,504 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,507 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000001
2016-05-17 20:45:46,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0, startTime=1463517946069, finishTime=1463517946500, timeTaken=431, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946451, Process_Duration=8, Process_End_Time=1463517946459, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:46,508 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:46,508 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:46,512 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,514 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:46,517 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_FINISHED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, taskId=task_1463517944674_0001_1_00_000000, startTime=1463517946069, finishTime=1463517946513, timeTaken=444, status=SUCCEEDED, successfulAttemptID=attempt_1463517944674_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946451, Process_Duration=8, Process_End_Time=1463517946459, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:46,520 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:46,520 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:46,524 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] : 1
2016-05-17 20:45:46,524 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:46,541 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:46,545 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_FINISHED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, vertexId=vertex_1463517944674_0001_1_00, initRequestedTime=1463517945723, initedTime=1463517945870, startRequestedTime=1463517945944, startedTime=1463517945944, finishTime=1463517946525, timeTaken=581, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946451, Process_Duration=8, Process_End_Time=1463517946459, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463517946069, firstTasksToStart=[ task_1463517944674_0001_1_00_000000 ], lastTaskFinishTime=1463517946500, lastTasksToFinish=[ task_1463517944674_0001_1_00_000000 ], minTaskDuration=431, maxTaskDuration=431, avgTaskDuration=431.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517944674_0001_1_00_000000 ], longestDurationTasks=[ task_1463517944674_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:46,546 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:46,547 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:46,548 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517944674_0001_00_000001, stoppedTime=1463517946547, exitStatus=0
2016-05-17 20:45:46,549 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:46,550 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] attempt: attempt_1463517944674_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:46,550 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,551 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:46,553 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:46,554 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517944674_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:46,554 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:46,554 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_STARTED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, taskId=task_1463517944674_0001_1_01_000000, scheduledTime=1463517946554, launchTime=1463517946554
2016-05-17 20:45:46,555 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:46,555 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:46,556 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:46,557 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: B99DE4A7E23B4D85A8EE364452481C90, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B7CF60ABA0644A7FA70974D7766572AB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA216359D1C94EB2B0C8F756359FEFD6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:46,557 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:46,558 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:46,561 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517944674_0001_00_000002
2016-05-17 20:45:46,563 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:46,566 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517944674_0001_00_000002
2016-05-17 20:45:46,566 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:46,567 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:46,568 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517944674_0001_00_000002, launchTime=1463517946567
2016-05-17 20:45:46,569 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:46,569 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517944674_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:46,569 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000001: 62 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:46,569 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517944674_0001_00_000001, Exiting
2016-05-17 20:45:46,570 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517944674_0001_00_000001
2016-05-17 20:45:46,570 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517944674_0001_00_000001
2016-05-17 20:45:46,569 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000002
2016-05-17 20:45:46,573 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517944674_0001_1_01_000000_0] to container: [container_1463517944674_0001_00_000002]
2016-05-17 20:45:46,573 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:46,574 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517944674_0001_00_000002 given task: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,574 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46 Starting to run new task attempt: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,575 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517944674_0001_1_01_000000_0] started. Is using containerId: [container_1463517944674_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:46,576 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517944674_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:46,577 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:46,577 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517944674_0001_00_000001 completed successfully
2016-05-17 20:45:46,575 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,577 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0, startTime=1463517946574, containerId=container_1463517944674_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517944674_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:46,577 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,579 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:46,579 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:46,579 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:46,579 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:46,579 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:46,580 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: B99DE4A7E23B4D85A8EE364452481C90, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B7CF60ABA0644A7FA70974D7766572AB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA216359D1C94EB2B0C8F756359FEFD6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:46,580 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:46,586 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:46,591 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517944674_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:46,592 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:46,593 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:46,586 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,594 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:46,594 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,596 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B7CF60ABA0644A7FA70974D7766572AB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:46,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:46,599 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,602 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DA216359D1C94EB2B0C8F756359FEFD6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:46,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:46,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:46,608 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:46,616 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:46,617 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:46,617 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:46,627 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:46,627 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:46,628 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,629 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,633 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B99DE4A7E23B4D85A8EE364452481C90, ordinal: 1
2016-05-17 20:45:46,633 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B99DE4A7E23B4D85A8EE364452481C90, mem on start (mb), free: 105, total: 281, max: 672
2016-05-17 20:45:46,633 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,633 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:46,633 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:46,634 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:46,634 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:46,634 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:46,638 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:46,639 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:46,639 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 739036352, TotalRequestedScaled: 3.69518176E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:1.05
2016-05-17 20:45:46,640 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DA216359D1C94EB2B0C8F756359FEFD6, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=66985083
2016-05-17 20:45:46,640 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B7CF60ABA0644A7FA70974D7766572AB, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=405125774
2016-05-17 20:45:46,640 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:46,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B7CF60ABA0644A7FA70974D7766572AB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:46,647 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:46,648 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:46,652 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:46,653 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:46,656 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B7CF60ABA0644A7FA70974D7766572AB with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:46,658 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=405125774. Updated to: ShuffleMem=405125774, postMergeMem: 0
2016-05-17 20:45:46,658 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=405125774, maxSingleShuffleLimit=101281440, mergeThreshold=364613184, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:46,662 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:46,666 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,666 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:46,667 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,669 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517944674_0001_1_00_000000_0_10002, runDuration: 326337, ]
2016-05-17 20:45:46,672 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:46,678 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517944674_0001_1_00_000000_0_10002] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463517946678, TimeTaken=2, Rate=0.10 MB/s
2016-05-17 20:45:46,679 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,680 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: F5ADF73F391F48779399EEE33FA9E9C0
2016-05-17 20:45:46,681 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 9EE434D1716D4BCC80D15007F33A09C4
2016-05-17 20:45:46,681 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:46,681 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B7CF60ABA0644A7FA70974D7766572AB] #1 in 6ms
2016-05-17 20:45:46,681 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 9EE434D1716D4BCC80D15007F33A09C4
2016-05-17 20:45:46,684 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3a6298b2; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1f16ec10; comparator=cascading.tuple.hadoop.util.TupleComparator@19e41e62; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:46,685 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:46,685 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:46,685 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:46,685 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B7CF60ABA0644A7FA70974D7766572AB] #1, status:false, isInterrupted:false
2016-05-17 20:45:46,748 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 16515068(66060272)
2016-05-17 20:45:46,751 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 63
2016-05-17 20:45:46,751 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:46,751 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 52848232
2016-05-17 20:45:46,753 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 66060288
2016-05-17 20:45:46,753 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 16515068; length = 4128768
2016-05-17 20:45:46,754 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] F5ADF73F391F48779399EEE33FA9E9C0, for 1 inputs
2016-05-17 20:45:46,757 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 20:45:46,759 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:46,763 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:46,768 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 20:45:46,769 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,770 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B99DE4A7E23B4D85A8EE364452481C90, all 1 inputs ready in: 00:00:00.015
2016-05-17 20:45:46,773 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:46,775 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@73778636; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@2c0c7e73
2016-05-17 20:45:46,786 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,786 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B99DE4A7E23B4D85A8EE364452481C90, mem on close (mb), free: 99, total: 345, max: 672
2016-05-17 20:45:46,786 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,787 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:46,788 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:46,788 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:46,789 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:46,789 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:46,790 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:46,791 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:46,791 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 184; bufvoid = 66060288
2016-05-17 20:45:46,791 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 16515068(66060272); kvend = 16515040(66060160); length = 29/4128768
2016-05-17 20:45:46,796 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:46 Completed running task attempt: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,802 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:46,803 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517944674_0001_1_01, eventCount=1
2016-05-17 20:45:46,804 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:46,809 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0, startTime=1463517946574, finishTime=1463517946803, timeTaken=229, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=28, MERGE_PHASE_TIME=116, FIRST_EVENT_RECEIVED=22, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946770, Process_Duration=16, Process_End_Time=1463517946786, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0
2016-05-17 20:45:46,809 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:46,809 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,810 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:46,811 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:46,813 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_FINISHED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, taskId=task_1463517944674_0001_1_01_000000, startTime=1463517946574, finishTime=1463517946809, timeTaken=235, status=SUCCEEDED, successfulAttemptID=attempt_1463517944674_0001_1_01_000000_0, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=28, MERGE_PHASE_TIME=116, FIRST_EVENT_RECEIVED=22, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946770, Process_Duration=16, Process_End_Time=1463517946786, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0
2016-05-17 20:45:46,813 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:46,814 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] : 1
2016-05-17 20:45:46,814 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:46,815 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:46,816 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 49 [[File System Counters FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=28, MERGE_PHASE_TIME=116, FIRST_EVENT_RECEIVED=22, LAST_EVENT_RECEIVED=22][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517946770, Process_Duration=16, Process_End_Time=1463517946786, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0]]
2016-05-17 20:45:46,819 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:46,819 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,821 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_FINISHED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, vertexId=vertex_1463517944674_0001_1_01, initRequestedTime=1463517945792, initedTime=1463517945851, startRequestedTime=1463517945986, startedTime=1463517945986, finishTime=1463517946814, timeTaken=828, status=SUCCEEDED, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=28, MERGE_PHASE_TIME=116, FIRST_EVENT_RECEIVED=22, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946770, Process_Duration=16, Process_End_Time=1463517946786, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, vertexStats=firstTaskStartTime=1463517946574, firstTasksToStart=[ task_1463517944674_0001_1_01_000000 ], lastTaskFinishTime=1463517946803, lastTasksToFinish=[ task_1463517944674_0001_1_01_000000 ], minTaskDuration=229, maxTaskDuration=229, avgTaskDuration=229.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517944674_0001_1_01_000000 ], longestDurationTasks=[ task_1463517944674_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:46,823 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:46,823 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:46,822 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000002
2016-05-17 20:45:46,825 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517944674_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:46,825 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:46,826 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517944674_0001_00_000002, Exiting
2016-05-17 20:45:46,826 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517944674_0001_00_000002
2016-05-17 20:45:46,826 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517944674_0001_00_000002
2016-05-17 20:45:46,827 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:46,827 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517944674_0001_00_000002, stoppedTime=1463517946824, exitStatus=0
2016-05-17 20:45:46,828 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:46,828 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517944674_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:46,828 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517944674_0001_00_000002 completed successfully
2016-05-17 20:45:46,828 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] attempt: attempt_1463517944674_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:46,829 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:46,829 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:46,830 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:46,830 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517944674_0001_00_000002
2016-05-17 20:45:46,830 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:46,831 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:46,832 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_STARTED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, taskId=task_1463517944674_0001_1_02_000000, scheduledTime=1463517946831, launchTime=1463517946831
2016-05-17 20:45:46,832 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:46,833 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:46,833 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: DA216359D1C94EB2B0C8F756359FEFD6, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A7B573490F9540059E3AEE82DB4B0EF9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:46,834 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:46,834 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:46,835 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517944674_0001_00_000003
2016-05-17 20:45:46,836 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:46,837 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517944674_0001_00_000003
2016-05-17 20:45:46,838 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:46,840 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:46,841 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517944674_0001_00_000003, launchTime=1463517946840
2016-05-17 20:45:46,840 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:46,842 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000003
2016-05-17 20:45:46,843 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517944674_0001_1_02_000000_0] to container: [container_1463517944674_0001_00_000003]
2016-05-17 20:45:46,843 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:46,843 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517944674_0001_00_000003 given task: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,844 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,844 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517944674_0001_1_02_000000_0] started. Is using containerId: [container_1463517944674_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 20:45:46,845 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:46,845 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0, startTime=1463517946844, containerId=container_1463517944674_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517944674_0001_00_000003/teamcity, completedLogs=
2016-05-17 20:45:46,846 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:46,846 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:46 Starting to run new task attempt: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,847 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,847 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,848 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:46,848 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:46,848 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:46,849 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:46,849 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:46,849 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: DA216359D1C94EB2B0C8F756359FEFD6, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A7B573490F9540059E3AEE82DB4B0EF9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:46,849 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:46,850 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,855 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:46,855 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:46,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:46,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A7B573490F9540059E3AEE82DB4B0EF9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:46,863 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:46,863 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:46,864 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,865 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,863 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:46,866 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A7B573490F9540059E3AEE82DB4B0EF9
2016-05-17 20:45:46,866 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:46,866 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A7B573490F9540059E3AEE82DB4B0EF9
2016-05-17 20:45:46,865 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:46,864 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517944674_0001_1_02_000000_0 sent events: (0-1)
2016-05-17 20:45:46,882 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DA216359D1C94EB2B0C8F756359FEFD6, ordinal: 2
2016-05-17 20:45:46,883 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA216359D1C94EB2B0C8F756359FEFD6, mem on start (mb), free: 93, total: 345, max: 672
2016-05-17 20:45:46,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,884 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:46,885 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:46,891 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:46,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A7B573490F9540059E3AEE82DB4B0EF9
2016-05-17 20:45:46,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:46,893 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:46,893 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:46,894 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:46,894 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:46,895 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:46,895 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:46,896 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B99DE4A7E23B4D85A8EE364452481C90, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:46,897 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A7B573490F9540059E3AEE82DB4B0EF9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:46,897 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:46,898 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B99DE4A7E23B4D85A8EE364452481C90 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:46,898 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,899 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:46,900 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:46,901 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:46,901 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:46,902 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B99DE4A7E23B4D85A8EE364452481C90 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:46,902 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:46,902 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:46,902 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:46,906 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,906 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:46,908 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,912 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517944674_0001_1_01_000000_0_10006, runDuration: 195730, ]
2016-05-17 20:45:46,913 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:46,914 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517944674_0001_1_01_000000_0_10006] to DISK_DIRECT, CompressedSize=210, DecompressedSize=206,EndTime=1463517946914, TimeTaken=1, Rate=0.20 MB/s
2016-05-17 20:45:46,915 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,916 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:46,916 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B99DE4A7E23B4D85A8EE364452481C90] #2 in 3ms
2016-05-17 20:45:46,919 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B99DE4A7E23B4D85A8EE364452481C90] #2, status:false, isInterrupted:false
2016-05-17 20:45:46,919 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 9EE434D1716D4BCC80D15007F33A09C4
2016-05-17 20:45:46,920 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"], id: A7B573490F9540059E3AEE82DB4B0EF9
2016-05-17 20:45:46,920 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-17 20:45:46,921 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:46,922 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 9EE434D1716D4BCC80D15007F33A09C4, for 1 inputs
2016-05-17 20:45:46,924 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 210 bytes from disk
2016-05-17 20:45:46,924 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:46,925 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:46,926 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2016-05-17 20:45:46,927 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,927 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DA216359D1C94EB2B0C8F756359FEFD6, all 1 inputs ready in: 00:00:00.005
2016-05-17 20:45:46,928 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:46,930 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@6d1be32; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@341dc81d
2016-05-17 20:45:46,934 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:46,935 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:46,936 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,937 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517944674_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 20:45:46,939 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,939 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA216359D1C94EB2B0C8F756359FEFD6, mem on close (mb), free: 89, total: 345, max: 672
2016-05-17 20:45:46,940 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,941 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:46,943 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:46,943 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:46,943 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:46,944 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:46 Completed running task attempt: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,946 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:46,948 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=26, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517946928, Process_Duration=11, Process_End_Time=1463517946939, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 20:45:46,948 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:46,949 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,950 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000003
2016-05-17 20:45:46,951 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0, startTime=1463517946844, finishTime=1463517946949, timeTaken=105, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=26, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946928, Process_Duration=11, Process_End_Time=1463517946939, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:46,950 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:46,952 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:46,954 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,955 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:46,956 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_FINISHED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, taskId=task_1463517944674_0001_1_02_000000, startTime=1463517946844, finishTime=1463517946955, timeTaken=111, status=SUCCEEDED, successfulAttemptID=attempt_1463517944674_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=26, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946928, Process_Duration=11, Process_End_Time=1463517946939, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:46,957 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:46,957 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] : 1
2016-05-17 20:45:46,957 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:46,958 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:46,960 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:46,963 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_FINISHED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, vertexId=vertex_1463517944674_0001_1_02, initRequestedTime=1463517945876, initedTime=1463517945942, startRequestedTime=1463517945998, startedTime=1463517945998, finishTime=1463517946957, timeTaken=959, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=26, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946928, Process_Duration=11, Process_End_Time=1463517946939, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463517946844, firstTasksToStart=[ task_1463517944674_0001_1_02_000000 ], lastTaskFinishTime=1463517946949, lastTasksToFinish=[ task_1463517944674_0001_1_02_000000 ], minTaskDuration=105, maxTaskDuration=105, avgTaskDuration=105.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517944674_0001_1_02_000000 ], longestDurationTasks=[ task_1463517944674_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:46,963 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:46,964 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:46,964 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517944674_0001_00_000003, stoppedTime=1463517946964, exitStatus=0
2016-05-17 20:45:46,965 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:46,965 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:46,966 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:46,967 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517944674_0001_1
2016-05-17 20:45:46,968 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517944674_0001_1
2016-05-17 20:45:46,968 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:46,969 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A7B573490F9540059E3AEE82DB4B0EF9 for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:46,969 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517944674_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 20:45:46,970 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000003: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:46,970 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517944674_0001_00_000003, Exiting
2016-05-17 20:45:46,970 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517944674_0001_00_000003
2016-05-17 20:45:46,970 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517944674_0001_00_000003
2016-05-17 20:45:46,972 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517944674_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:46,972 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517944674_0001_00_000003 completed successfully
2016-05-17 20:45:46,985 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:46,992 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517944674_0001_1, queueSize=0
2016-05-17 20:45:46,992 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:46,993 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517944674_0001_1, startTime=1463517945719, finishTime=1463517946985, timeTaken=1266, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2716, FILE_BYTES_WRITTEN=618, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=14, REDUCE_INPUT_RECORDS=18, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=36, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1018691584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=26, OUTPUT_BYTES=394, OUTPUT_BYTES_WITH_OVERHEAD=406, OUTPUT_BYTES_PHYSICAL=414, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=414, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=414, SHUFFLE_BYTES_DECOMPRESSED=406, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=414, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=43, MERGE_PHASE_TIME=142, FIRST_EVENT_RECEIVED=34, LAST_EVENT_RECEIVED=34, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390553840149, Process_Duration=35, Process_End_Time=4390553840184, Read_Duration=5, Tuples_Read=28, Tuples_Written=26, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 20:45:46,994 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517944674_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:46,994 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517944674_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:46,995 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517944674_0001_00_000003
2016-05-17 20:45:46,995 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:46 Completed Dag: dag_1463517944674_0001_1
2016-05-17 20:45:46,996 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517944674_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:46,996 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517944674_0001_1_post
2016-05-17 20:45:46,997 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,997 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:47,008 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463517944674_0001
2016-05-17 20:45:47,013 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:47,013 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:47,014 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:47,014 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:47,014 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:47,015 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:47,018 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:47,019 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:47,019 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:47,019 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:47,020 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:47,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:47,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:47,027 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001
2016-05-17 20:45:47,029 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain/_temporary
2016-05-17 20:45:47,030 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:02.535
2016-05-17 20:45:47,075 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-17 20:45:47,117 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:47,118 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:47,181 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 20:45:47,212 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.060
2016-05-17 20:45:47,214 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:47,214 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:47,214 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:47,233 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 20:45:47,245 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count
2016-05-17 20:45:47,348 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 20:45:47,348 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:47,348 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-17 20:45:47,348 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  parallel execution of steps is enabled: false
2016-05-17 20:45:47,348 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  executing total steps: 1
2016-05-17 20:45:47,348 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  allocating management threads: 1
2016-05-17 20:45:47,349 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...splatform/testCount/count
2016-05-17 20:45:47,350 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:47,371 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 20:45:47,377 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:47,378 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:47,378 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:47,391 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001 doesn't exist and is created
2016-05-17 20:45:47,410 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:47,410 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:47,412 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001
2016-05-17 20:45:47,416 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517947378_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:47,433 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:47,434 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:47,437 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:47,438 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:47,438 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001/localmode-log-dir
2016-05-17 20:45:47,438 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001/localmode-log-dir/history.txt.appattempt_1463517947378_0001_000000, maxErrors=10
2016-05-17 20:45:47,438 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:47,439 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517947378_0001_000000, appSubmitTime=1463517947416, launchTime=1463517947416
2016-05-17 20:45:47,444 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:47,445 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:47,447 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:47,451 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:52446
2016-05-17 20:45:47,452 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:47,455 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:47,457 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517947378_0001_000000, startTime=1463517947457
2016-05-17 20:45:47,457 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:47,457 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:47,511 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:47,512 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:47,512 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463517947378_0001, dagName=[79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count
2016-05-17 20:45:47,533 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count
2016-05-17 20:45:47,536 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517947378_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001/localmode-log-dir/dag_1463517947378_0001_1.dot
2016-05-17 20:45:47,538 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001/localmode-log-dir/dag_1463517947378_0001_1-tez-dag.pb.txt
2016-05-17 20:45:47,565 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517947378_0001_1
2016-05-17 20:45:47,566 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,566 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count
2016-05-17 20:45:47 Running Dag: dag_1463517947378_0001_1
2016-05-17 20:45:47,582 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517947378_0001_1, submitTime=1463517947533
2016-05-17 20:45:47,582 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:47,583 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,584 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:47,584 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517947378_0001_1, initTime=1463517947582
2016-05-17 20:45:47,584 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:47,585 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517947378_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:47,586 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_STARTED]: dagID=dag_1463517947378_0001_1, startTime=1463517947585
2016-05-17 20:45:47,586 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:47,586 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:47,586 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517947378_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:47,586 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E07BAAFDDE004DF18A0716FB41FA3255 : {EF1BAF328A81482DAB50C347146DF1C9={InputName=EF1BAF328A81482DAB50C347146DF1C9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:47,586 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EF1BAF328A81482DAB50C347146DF1C9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:47,586 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,587 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,587 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,587 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]. Starting root input initializers: 1
2016-05-17 20:45:47,588 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:47,588 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,590 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463517947378_0001, dagName=[79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count
2016-05-17 20:45:47,594 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:47,595 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,595 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,595 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,595 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2363FCD16E1448D4978D6CD5181A13C5, vertexId=vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:47,596 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2363FCD16E1448D4978D6CD5181A13C5, vertexId=vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,600 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EF1BAF328A81482DAB50C347146DF1C9 on vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,604 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:47,609 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EF1BAF328A81482DAB50C347146DF1C9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:47,609 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:47,609 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:47,607 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2363FCD16E1448D4978D6CD5181A13C5, vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:47,610 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2363FCD16E1448D4978D6CD5181A13C5, vertexId=vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,612 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:47,612 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, vertexId=vertex_1463517947378_0001_1_01, initRequestedTime=1463517947588, initedTime=1463517947611, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:47,613 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:47,615 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 20:45:47,619 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EF1BAF328A81482DAB50C347146DF1C9 on vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,619 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:47,620 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EF1BAF328A81482DAB50C347146DF1C9=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:47,621 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] parallelism set to 1
2016-05-17 20:45:47,621 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:47,622 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517947378_0001_1_00, eventCount=1
2016-05-17 20:45:47,629 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:47,630 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, vertexId=vertex_1463517947378_0001_1_00, initRequestedTime=1463517947586, initedTime=1463517947628, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:47,630 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] with distanceFromRoot: 0
2016-05-17 20:45:47,630 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:47,631 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,632 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,636 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:47,636 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463517947378_0001, with dag id: dag_1463517947378_0001_1
2016-05-17 20:45:47,637 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517947378_0001_1_00, startRequestedTime=1463517947631, startedTime=1463517947631
2016-05-17 20:45:47,638 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:47,640 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:47,640 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_STARTED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, taskId=task_1463517947378_0001_1_00_000000, scheduledTime=1463517947639, launchTime=1463517947639
2016-05-17 20:45:47,641 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:47,641 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517947378_0001_1_00 for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] numStartedSources: 1 numSources: 1
2016-05-17 20:45:47,641 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] with distanceFromRoot: 1
2016-05-17 20:45:47,643 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,644 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E07BAAFDDE004DF18A0716FB41FA3255 in vertex: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,644 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EE2A5BED0ACD4A2DBB81B997AAE33564 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:47,645 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:47,645 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517947378_0001_1_01, startRequestedTime=1463517947644, startedTime=1463517947644
2016-05-17 20:45:47,646 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:47,649 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count, VertexName: E07BAAFDDE004DF18A0716FB41FA3255, VertexParallelism: 1, TaskAttemptID:attempt_1463517947378_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF1BAF328A81482DAB50C347146DF1C9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:47,651 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:47,652 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:47,655 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:47,656 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:47,657 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:47,658 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517947378_0001_00_000001
2016-05-17 20:45:47,659 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:47,664 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517947378_0001_00_000001
2016-05-17 20:45:47,664 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:47,665 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:47,668 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517947378_0001_00_000001, launchTime=1463517947665
2016-05-17 20:45:47,668 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517947378_0001_00_000001
2016-05-17 20:45:47,669 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:47,670 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517947378_0001_1_00_000000_0] to container: [container_1463517947378_0001_00_000001]
2016-05-17 20:45:47,671 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:47,672 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517947378_0001_00_000001 given task: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,672 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517947378_0001_1_00_000000_0] started. Is using containerId: [container_1463517947378_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:47,674 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0, startTime=1463517947672, containerId=container_1463517947378_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517947378_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:47,674 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:47,676 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:47,676 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:47,677 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517947378_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47 Starting to run new task attempt: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,678 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,678 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,679 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:47,679 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:47,679 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:47,679 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:47,679 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:47,679 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count, VertexName: E07BAAFDDE004DF18A0716FB41FA3255, VertexParallelism: 1, TaskAttemptID:attempt_1463517947378_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF1BAF328A81482DAB50C347146DF1C9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:47,680 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:47,683 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,686 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:47,687 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,689 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517947378_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:47,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EF1BAF328A81482DAB50C347146DF1C9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:47,691 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:47,692 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,692 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,696 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,696 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:47,696 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:47,697 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,699 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:47,699 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,701 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,702 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:47,705 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:47,706 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,706 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,716 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E07BAAFDDE004DF18A0716FB41FA3255, ordinal: 0
2016-05-17 20:45:47,716 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E07BAAFDDE004DF18A0716FB41FA3255, mem on start (mb), free: 214, total: 461, max: 672
2016-05-17 20:45:47,716 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:47,717 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:47,718 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:47,718 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:47,719 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:47,719 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:47,720 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:47,720 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:47,721 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:47,722 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EF1BAF328A81482DAB50C347146DF1C9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:47,722 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EE2A5BED0ACD4A2DBB81B997AAE33564, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:47,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:47,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,723 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:47,723 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,725 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EF1BAF328A81482DAB50C347146DF1C9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:47,726 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:47,729 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:47,729 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:47,730 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,735 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:47,737 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:47,737 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@21805932
2016-05-17 20:45:47,738 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,739 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 9019254DB36E43CC8F5BF866381A09A3
2016-05-17 20:45:47,739 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 9019254DB36E43CC8F5BF866381A09A3
2016-05-17 20:45:47,743 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@21805932
2016-05-17 20:45:47,743 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:47,743 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:47,745 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@622f680b; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1657dc89; comparator=cascading.tuple.hadoop.util.TupleComparator@624fb1d4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:47,746 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:47,746 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:47,746 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:47,787 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:47,787 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:47,787 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:47,788 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:47,788 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:47,789 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:47,789 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:47,790 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E07BAAFDDE004DF18A0716FB41FA3255, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:47,793 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,793 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E07BAAFDDE004DF18A0716FB41FA3255, mem on close (mb), free: 111, total: 461, max: 672
2016-05-17 20:45:47,793 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:47,793 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:47,794 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 20:45:47,794 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 20:45:47,801 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:47 Completed running task attempt: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,805 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:47,806 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517947790, Process_Duration=3, Process_End_Time=1463517947793, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 20:45:47,806 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517947378_0001_1_00, eventCount=1
2016-05-17 20:45:47,807 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:47,806 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:47,808 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,808 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0, startTime=1463517947672, finishTime=1463517947807, timeTaken=135, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947790, Process_Duration=3, Process_End_Time=1463517947793, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:47,809 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:47,809 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:47,810 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517947378_0001_00_000001
2016-05-17 20:45:47,811 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,811 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:47,811 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_FINISHED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, taskId=task_1463517947378_0001_1_00_000000, startTime=1463517947672, finishTime=1463517947811, timeTaken=139, status=SUCCEEDED, successfulAttemptID=attempt_1463517947378_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947790, Process_Duration=3, Process_End_Time=1463517947793, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:47,813 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:47,813 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] : 1
2016-05-17 20:45:47,813 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,815 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:47,817 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_FINISHED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, vertexId=vertex_1463517947378_0001_1_00, initRequestedTime=1463517947586, initedTime=1463517947628, startRequestedTime=1463517947631, startedTime=1463517947631, finishTime=1463517947813, timeTaken=182, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947790, Process_Duration=3, Process_End_Time=1463517947793, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463517947672, firstTasksToStart=[ task_1463517947378_0001_1_00_000000 ], lastTaskFinishTime=1463517947807, lastTasksToFinish=[ task_1463517947378_0001_1_00_000000 ], minTaskDuration=135, maxTaskDuration=135, avgTaskDuration=135.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517947378_0001_1_00_000000 ], longestDurationTasks=[ task_1463517947378_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:47,818 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:47,818 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:47,819 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:47,819 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517947378_0001_00_000001, stoppedTime=1463517947819, exitStatus=0
2016-05-17 20:45:47,821 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:47,821 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] attempt: attempt_1463517947378_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:47,822 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,822 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,823 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:47,823 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517947378_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:47,825 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:47,825 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517947378_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:47,825 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517947378_0001_00_000001: 15 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:47,825 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_STARTED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, taskId=task_1463517947378_0001_1_01_000000, scheduledTime=1463517947825, launchTime=1463517947825
2016-05-17 20:45:47,826 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:47,826 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:47,826 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517947378_0001_00_000001, Exiting
2016-05-17 20:45:47,826 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517947378_0001_00_000001
2016-05-17 20:45:47,826 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517947378_0001_00_000001
2016-05-17 20:45:47,830 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517947378_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:47,832 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517947378_0001_00_000001 completed successfully
2016-05-17 20:45:47,830 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,833 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:47,833 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count, VertexName: EE2A5BED0ACD4A2DBB81B997AAE33564, VertexParallelism: 1, TaskAttemptID:attempt_1463517947378_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E07BAAFDDE004DF18A0716FB41FA3255, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2363FCD16E1448D4978D6CD5181A13C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:47,833 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:47,834 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:47,835 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517947378_0001_00_000002
2016-05-17 20:45:47,836 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:47,840 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517947378_0001_00_000002
2016-05-17 20:45:47,841 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:47,846 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:47,846 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:47,847 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517947378_0001_00_000002, launchTime=1463517947845
2016-05-17 20:45:47,848 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517947378_0001_00_000002
2016-05-17 20:45:47,849 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517947378_0001_1_01_000000_0] to container: [container_1463517947378_0001_00_000002]
2016-05-17 20:45:47,850 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:47,850 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517947378_0001_00_000002 given task: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,850 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517947378_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,851 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517947378_0001_1_01_000000_0] started. Is using containerId: [container_1463517947378_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:47 Starting to run new task attempt: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,851 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,851 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0, startTime=1463517947850, containerId=container_1463517947378_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517947378_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:47,851 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:47,852 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,852 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:47,852 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:47,852 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:47,852 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:47,852 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:47,852 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:47,853 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:47,853 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count, VertexName: EE2A5BED0ACD4A2DBB81B997AAE33564, VertexParallelism: 1, TaskAttemptID:attempt_1463517947378_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E07BAAFDDE004DF18A0716FB41FA3255, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2363FCD16E1448D4978D6CD5181A13C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:47,853 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:47,854 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,855 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:47,855 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,856 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517947378_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:47,857 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E07BAAFDDE004DF18A0716FB41FA3255, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:47,857 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2363FCD16E1448D4978D6CD5181A13C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:47,857 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:47,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:47,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2363FCD16E1448D4978D6CD5181A13C5
2016-05-17 20:45:47,861 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:47,861 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2363FCD16E1448D4978D6CD5181A13C5
2016-05-17 20:45:47,865 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:47,869 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:47,869 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:47,869 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,870 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,882 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EE2A5BED0ACD4A2DBB81B997AAE33564, ordinal: 1
2016-05-17 20:45:47,882 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE2A5BED0ACD4A2DBB81B997AAE33564, mem on start (mb), free: 102, total: 461, max: 672
2016-05-17 20:45:47,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:47,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:47,890 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:47,890 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2363FCD16E1448D4978D6CD5181A13C5
2016-05-17 20:45:47,891 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:47,891 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:47,891 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:47,891 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:47,891 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:47,891 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:47,892 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:47,892 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2363FCD16E1448D4978D6CD5181A13C5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:47,892 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E07BAAFDDE004DF18A0716FB41FA3255, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:47,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:47,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E07BAAFDDE004DF18A0716FB41FA3255 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:47,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:47,896 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:47,896 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:47,897 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:47,897 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E07BAAFDDE004DF18A0716FB41FA3255 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:47,897 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:47,897 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:47,898 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:47,900 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,900 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:47,900 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,901 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517947378_0001_1_00_000000_0_10012, runDuration: 106546, ]
2016-05-17 20:45:47,903 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517947378_0001_1_00_000000_0_10012] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463517947903, TimeTaken=2, Rate=0.22 MB/s
2016-05-17 20:45:47,903 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,904 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:47,904 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E07BAAFDDE004DF18A0716FB41FA3255] #3 in 3ms
2016-05-17 20:45:47,905 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E07BAAFDDE004DF18A0716FB41FA3255] #3, status:false, isInterrupted:false
2016-05-17 20:45:47,907 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:47,908 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 20:45:47,908 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:47,908 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:47,909 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 20:45:47,909 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,911 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:47,912 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 20:45:47,914 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 9019254DB36E43CC8F5BF866381A09A3
2016-05-17 20:45:47,914 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"], id: 2363FCD16E1448D4978D6CD5181A13C5
2016-05-17 20:45:47,915 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-17 20:45:47,915 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 9019254DB36E43CC8F5BF866381A09A3, for 1 inputs
2016-05-17 20:45:47,916 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EE2A5BED0ACD4A2DBB81B997AAE33564, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:47,918 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5eb3589d; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@507edbfd
2016-05-17 20:45:47,923 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:47,924 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:47,927 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,928 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517947378_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:47,929 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,929 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE2A5BED0ACD4A2DBB81B997AAE33564, mem on close (mb), free: 97, total: 461, max: 672
2016-05-17 20:45:47,929 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,929 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:47,930 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:47,930 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:47,930 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:47,930 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:47 Completed running task attempt: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,931 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:47,935 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:47,937 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0, startTime=1463517947850, finishTime=1463517947934, timeTaken=84, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947916, Process_Duration=13, Process_End_Time=1463517947929, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:47,937 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:47,937 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:47,937 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,938 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:47,939 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_FINISHED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, taskId=task_1463517947378_0001_1_01_000000, startTime=1463517947850, finishTime=1463517947938, timeTaken=88, status=SUCCEEDED, successfulAttemptID=attempt_1463517947378_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947916, Process_Duration=13, Process_End_Time=1463517947929, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:47,939 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:47,940 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] : 1
2016-05-17 20:45:47,940 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,941 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517947916, Process_Duration=13, Process_End_Time=1463517947929, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 20:45:47,942 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:47,942 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,943 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517947378_0001_00_000002
2016-05-17 20:45:47,943 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:47,945 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_FINISHED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, vertexId=vertex_1463517947378_0001_1_01, initRequestedTime=1463517947588, initedTime=1463517947611, startRequestedTime=1463517947644, startedTime=1463517947644, finishTime=1463517947940, timeTaken=296, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947916, Process_Duration=13, Process_End_Time=1463517947929, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463517947850, firstTasksToStart=[ task_1463517947378_0001_1_01_000000 ], lastTaskFinishTime=1463517947934, lastTasksToFinish=[ task_1463517947378_0001_1_01_000000 ], minTaskDuration=84, maxTaskDuration=84, avgTaskDuration=84.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517947378_0001_1_01_000000 ], longestDurationTasks=[ task_1463517947378_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:47,945 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:47,945 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:47,946 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517947378_0001_00_000002, stoppedTime=1463517947946, exitStatus=0
2016-05-17 20:45:47,946 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:47,946 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:47,947 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:47,947 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517947378_0001_1
2016-05-17 20:45:47,948 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517947378_0001_1
2016-05-17 20:45:47,948 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,948 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2363FCD16E1448D4978D6CD5181A13C5 for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,953 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517947378_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:47,954 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517947378_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:47,954 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517947378_0001_00_000002, Exiting
2016-05-17 20:45:47,954 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517947378_0001_00_000002
2016-05-17 20:45:47,954 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517947378_0001_00_000002
2016-05-17 20:45:47,955 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517947378_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:47,955 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517947378_0001_00_000002 completed successfully
2016-05-17 20:45:47,964 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517947378_0001_1, queueSize=0
2016-05-17 20:45:47,964 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:47,966 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517947378_0001_1, startTime=1463517947585, finishTime=1463517947960, timeTaken=375, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=966787072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035895706, Process_Duration=16, Process_End_Time=2927035895722, Read_Duration=4, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 20:45:47,966 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517947378_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:47,966 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517947378_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:47,966 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:47,967 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517947378_0001_00_000002
2016-05-17 20:45:47 Completed Dag: dag_1463517947378_0001_1
2016-05-17 20:45:47,967 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517947378_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:47,968 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517947378_0001_1_post
2016-05-17 20:45:47,968 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,968 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:47,978 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463517947378_0001
2016-05-17 20:45:47,981 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:47,982 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:47,982 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:47,982 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:47,982 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:47,983 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:47,983 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:47,983 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:47,984 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:47,984 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:47,983 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:47,984 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:47,986 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:47,991 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001
2016-05-17 20:45:47,991 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count/_temporary
2016-05-17 20:45:47,992 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  completed in: 00:00.643
2016-05-17 20:45:48,089 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount/count
2016-05-17 20:45:48,129 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:48,129 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:48,193 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 20:45:48,208 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.037
2016-05-17 20:45:48,209 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:48,209 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:48,210 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:48,232 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:48,233 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-17 20:45:48,245 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:48,245 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:48,246 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-17 20:45:48,246 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:48,246 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:48,246 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:48,247 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48,247 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:48,267 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:48,272 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:48,272 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:48,273 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:48,282 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001 doesn't exist and is created
2016-05-17 20:45:48,303 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:48,303 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:48,305 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001
2016-05-17 20:45:48,310 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517948272_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:48,326 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:48,327 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:48,329 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:48,329 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:48,330 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001/localmode-log-dir
2016-05-17 20:45:48,330 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001/localmode-log-dir/history.txt.appattempt_1463517948272_0001_000000, maxErrors=10
2016-05-17 20:45:48,330 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:48,331 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517948272_0001_000000, appSubmitTime=1463517948309, launchTime=1463517948309
2016-05-17 20:45:48,361 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:48,369 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44085
2016-05-17 20:45:48,371 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:48,373 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:48,373 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:48,374 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:48,377 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517948272_0001_000000, startTime=1463517948377
2016-05-17 20:45:48,377 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:48,377 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:48,404 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:48,404 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:48,404 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463517948272_0001, dagName=[EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48,425 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48,427 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517948272_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001/localmode-log-dir/dag_1463517948272_0001_1.dot
2016-05-17 20:45:48,428 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001/localmode-log-dir/dag_1463517948272_0001_1-tez-dag.pb.txt
2016-05-17 20:45:48,451 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517948272_0001_1
2016-05-17 20:45:48,451 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,451 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48 Running Dag: dag_1463517948272_0001_1
2016-05-17 20:45:48,464 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517948272_0001_1, submitTime=1463517948425
2016-05-17 20:45:48,464 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D9053C7DA5184C128BE4DC7D95419D4D
2016-05-17 20:45:48,465 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:48,465 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:48,466 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:48,466 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517948272_0001_1, initTime=1463517948464
2016-05-17 20:45:48,466 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948272_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:48,467 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_STARTED]: dagID=dag_1463517948272_0001_1, startTime=1463517948467
2016-05-17 20:45:48,467 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:48,467 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948272_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:48,468 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D9053C7DA5184C128BE4DC7D95419D4D : {87C55A64C93A44A2ABBD366B2E9FA4DC={InputName=87C55A64C93A44A2ABBD366B2E9FA4DC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:48,468 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 87C55A64C93A44A2ABBD366B2E9FA4DC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:48,468 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,468 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,470 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,467 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:48,470 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]. Starting root input initializers: 1
2016-05-17 20:45:48,471 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:48,472 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463517948272_0001, dagName=[EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48,472 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 87C55A64C93A44A2ABBD366B2E9FA4DC on vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,474 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:48,474 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 87C55A64C93A44A2ABBD366B2E9FA4DC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:48,475 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:48,475 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:48,478 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:48,478 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 87C55A64C93A44A2ABBD366B2E9FA4DC on vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,479 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:48,479 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {87C55A64C93A44A2ABBD366B2E9FA4DC=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:48,479 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] parallelism set to 1
2016-05-17 20:45:48,479 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:48,480 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517948272_0001_1_00, eventCount=1
2016-05-17 20:45:48,480 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,481 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F41FA8993CFD46A180CF5ADC68F147B2, vertexId=vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:48,481 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F41FA8993CFD46A180CF5ADC68F147B2, vertexId=vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,483 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F41FA8993CFD46A180CF5ADC68F147B2, vertexName=D9053C7DA5184C128BE4DC7D95419D4D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:48,483 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F41FA8993CFD46A180CF5ADC68F147B2, vertexId=vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,485 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:48,485 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, vertexId=vertex_1463517948272_0001_1_00, initRequestedTime=1463517948468, initedTime=1463517948485, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:48,485 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] with distanceFromRoot: 0
2016-05-17 20:45:48,486 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:48,486 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D9053C7DA5184C128BE4DC7D95419D4D
2016-05-17 20:45:48,486 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,486 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:48,486 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517948272_0001_1_00, startRequestedTime=1463517948486, startedTime=1463517948486
2016-05-17 20:45:48,487 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:48,488 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:48,488 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:TASK_STARTED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, taskId=task_1463517948272_0001_1_00_000000, scheduledTime=1463517948487, launchTime=1463517948487
2016-05-17 20:45:48,488 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948272_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:48,488 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,489 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap, VertexName: D9053C7DA5184C128BE4DC7D95419D4D, VertexParallelism: 1, TaskAttemptID:attempt_1463517948272_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=87C55A64C93A44A2ABBD366B2E9FA4DC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F41FA8993CFD46A180CF5ADC68F147B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:48,489 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948272_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:48,490 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:48,491 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:48,492 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:48,497 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:48,497 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517948272_0001_00_000001
2016-05-17 20:45:48,497 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:48,500 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517948272_0001_00_000001
2016-05-17 20:45:48,500 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:48,503 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:48,503 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:48,503 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517948272_0001, with dag id: dag_1463517948272_0001_1
2016-05-17 20:45:48,504 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517948272_0001_00_000001, launchTime=1463517948502
2016-05-17 20:45:48,506 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517948272_0001_00_000001
2016-05-17 20:45:48,510 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517948272_0001_1_00_000000_0] to container: [container_1463517948272_0001_00_000001]
2016-05-17 20:45:48,510 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:48,510 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517948272_0001_00_000001 given task: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,511 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517948272_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48 Starting to run new task attempt: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,512 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,512 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,512 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517948272_0001_1_00_000000_0] started. Is using containerId: [container_1463517948272_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:48,512 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:48,512 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0, startTime=1463517948512, containerId=container_1463517948272_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517948272_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:48,513 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:48,512 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:48,513 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:48,513 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:48,514 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:48,514 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap, VertexName: D9053C7DA5184C128BE4DC7D95419D4D, VertexParallelism: 1, TaskAttemptID:attempt_1463517948272_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=87C55A64C93A44A2ABBD366B2E9FA4DC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F41FA8993CFD46A180CF5ADC68F147B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:48,514 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:48,513 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948272_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:48,515 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517948272_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:48,516 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948272_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:48,516 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,516 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:48,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:48,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=87C55A64C93A44A2ABBD366B2E9FA4DC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:48,518 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:48,518 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,518 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:48,523 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:48,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F41FA8993CFD46A180CF5ADC68F147B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:48,525 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:48,525 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:48,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,534 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F41FA8993CFD46A180CF5ADC68F147B2
2016-05-17 20:45:48,534 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:48,534 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F41FA8993CFD46A180CF5ADC68F147B2
2016-05-17 20:45:48,541 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D9053C7DA5184C128BE4DC7D95419D4D, ordinal: 0
2016-05-17 20:45:48,541 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9053C7DA5184C128BE4DC7D95419D4D, mem on start (mb), free: 239, total: 461, max: 672
2016-05-17 20:45:48,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:48,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:48,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:48,546 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:48,546 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F41FA8993CFD46A180CF5ADC68F147B2
2016-05-17 20:45:48,547 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:48,547 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:48,547 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:48,548 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:48,548 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:48,548 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:48,548 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:48,549 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:48,549 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 87C55A64C93A44A2ABBD366B2E9FA4DC, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:48,549 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F41FA8993CFD46A180CF5ADC68F147B2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:48,549 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:48,550 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,550 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:48,550 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,550 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 87C55A64C93A44A2ABBD366B2E9FA4DC being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:48,550 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:48,551 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:48,551 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:48,552 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,552 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:48,553 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@360b662b
2016-05-17 20:45:48,556 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@360b662b
2016-05-17 20:45:48,556 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:48,557 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:48,557 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:48,558 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,559 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"], id: F41FA8993CFD46A180CF5ADC68F147B2
2016-05-17 20:45:48,559 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-17 20:45:48,560 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:48,560 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:48,560 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D9053C7DA5184C128BE4DC7D95419D4D, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:48,563 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:48,563 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:48,564 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,564 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517948272_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:48,565 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,565 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9053C7DA5184C128BE4DC7D95419D4D, mem on close (mb), free: 237, total: 461, max: 672
2016-05-17 20:45:48 Completed running task attempt: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,567 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:48,568 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0, startTime=1463517948512, finishTime=1463517948568, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:48,569 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:48,569 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948272_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:48,570 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:48,570 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,571 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:48,571 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:TASK_FINISHED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, taskId=task_1463517948272_0001_1_00_000000, startTime=1463517948512, finishTime=1463517948570, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1463517948272_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:48,571 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948272_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:48,571 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] : 1
2016-05-17 20:45:48,571 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,572 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-17 20:45:48,573 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:VERTEX_FINISHED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, vertexId=vertex_1463517948272_0001_1_00, initRequestedTime=1463517948468, initedTime=1463517948485, startRequestedTime=1463517948486, startedTime=1463517948486, finishTime=1463517948571, timeTaken=85, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463517948512, firstTasksToStart=[ task_1463517948272_0001_1_00_000000 ], lastTaskFinishTime=1463517948568, lastTasksToFinish=[ task_1463517948272_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517948272_0001_1_00_000000 ], longestDurationTasks=[ task_1463517948272_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:48,574 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:48,574 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:48,573 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:48,576 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517948272_0001_00_000001, stoppedTime=1463517948576, exitStatus=0
2016-05-17 20:45:48,576 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:48,577 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:48,577 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517948272_0001_1
2016-05-17 20:45:48,578 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517948272_0001_1
2016-05-17 20:45:48,578 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F41FA8993CFD46A180CF5ADC68F147B2 for vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,579 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:48,579 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,576 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:48,580 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517948272_0001_00_000001
2016-05-17 20:45:48,582 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517948272_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:48,582 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517948272_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:48,583 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517948272_0001_00_000001, Exiting
2016-05-17 20:45:48,583 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517948272_0001_00_000001
2016-05-17 20:45:48,583 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517948272_0001_00_000001
2016-05-17 20:45:48,585 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517948272_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:48,586 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517948272_0001_00_000001 completed successfully
2016-05-17 20:45:48,591 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517948272_0001_1, queueSize=0
2016-05-17 20:45:48,592 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517948272_0001_1, startTime=1463517948467, finishTime=1463517948590, timeTaken=123, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:48,592 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517948272_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:48,592 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948272_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:48,592 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:48,592 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517948272_0001_00_000001
2016-05-17 20:45:48 Completed Dag: dag_1463517948272_0001_1
2016-05-17 20:45:48,592 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517948272_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:48,593 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517948272_0001_1_post
2016-05-17 20:45:48,593 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,593 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:48,593 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:48,601 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463517948272_0001
2016-05-17 20:45:48,603 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:48,603 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:48,603 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:48,603 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:48,603 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:48,604 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:48,604 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:48,605 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:48,605 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:48,606 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:48,606 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:48,607 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:48,607 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:48,615 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001
2016-05-17 20:45:48,616 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap/_temporary
2016-05-17 20:45:48,617 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.371
2016-05-17 20:45:48,656 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-17 20:45:48,685 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:48,685 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:48,729 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 20:45:48,741 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-17 20:45:48,742 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:48,742 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:48,743 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:48,760 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt
2016-05-17 20:45:48,761 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-17 20:45:48,776 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:48,777 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 20:45:48,777 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-17 20:45:48,777 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:48,777 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:48,778 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:48,779 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:48,779 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:48,795 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:48,800 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:48,800 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:48,800 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:48,811 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001 doesn't exist and is created
2016-05-17 20:45:48,834 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:48,834 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:48,835 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001
2016-05-17 20:45:48,840 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517948800_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:48,852 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:48,853 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:48,855 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:48,855 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:48,855 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001/localmode-log-dir
2016-05-17 20:45:48,855 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001/localmode-log-dir/history.txt.appattempt_1463517948800_0001_000000, maxErrors=10
2016-05-17 20:45:48,856 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:48,856 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517948800_0001_000000, appSubmitTime=1463517948839, launchTime=1463517948839
2016-05-17 20:45:48,861 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:48,863 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:48,865 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:48,867 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:48,869 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38845
2016-05-17 20:45:48,871 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:48,874 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517948800_0001_000000, startTime=1463517948874
2016-05-17 20:45:48,874 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:48,874 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:48,934 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:48,934 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:48,935 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463517948800_0001, dagName=[47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:48,957 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:48,960 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517948800_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001/localmode-log-dir/dag_1463517948800_0001_1.dot
2016-05-17 20:45:48,961 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001/localmode-log-dir/dag_1463517948800_0001_1-tez-dag.pb.txt
2016-05-17 20:45:48,990 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517948800_0001_1
2016-05-17 20:45:48,991 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,991 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:48 Running Dag: dag_1463517948800_0001_1
2016-05-17 20:45:49,004 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:49,004 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517948800_0001_1, submitTime=1463517948957
2016-05-17 20:45:49,005 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F8590CB2C4D643849371443AE7B4080B
2016-05-17 20:45:49,005 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:49,006 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:49,006 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517948800_0001_1, initTime=1463517949005
2016-05-17 20:45:49,006 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948800_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:49,007 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_STARTED]: dagID=dag_1463517948800_0001_1, startTime=1463517949007
2016-05-17 20:45:49,007 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:49,007 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948800_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:49,007 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F8590CB2C4D643849371443AE7B4080B : {70DA86223CD7470C90C12B19E2874E00={InputName=70DA86223CD7470C90C12B19E2874E00}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:49,007 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:49,007 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 70DA86223CD7470C90C12B19E2874E00, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:49,008 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,008 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,008 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,008 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]. Starting root input initializers: 1
2016-05-17 20:45:49,009 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:49,010 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 70DA86223CD7470C90C12B19E2874E00 on vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,010 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463517948800_0001, dagName=[47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:49,014 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:49,014 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 70DA86223CD7470C90C12B19E2874E00 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:49,015 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:49,015 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:49,018 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 412
2016-05-17 20:45:49,021 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 70DA86223CD7470C90C12B19E2874E00 on vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,021 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,021 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {70DA86223CD7470C90C12B19E2874E00=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:49,022 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] parallelism set to 1
2016-05-17 20:45:49,022 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,022 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517948800_0001_1_00, eventCount=1
2016-05-17 20:45:49,023 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,024 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D03628992BF34FA1A5488F67378A92C5, vertexId=vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:49,024 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D03628992BF34FA1A5488F67378A92C5, vertexId=vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,026 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D03628992BF34FA1A5488F67378A92C5, vertexName=F8590CB2C4D643849371443AE7B4080B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:49,026 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D03628992BF34FA1A5488F67378A92C5, vertexId=vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,028 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F8590CB2C4D643849371443AE7B4080B, vertexId=vertex_1463517948800_0001_1_00, initRequestedTime=1463517949007, initedTime=1463517949028, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:49,028 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] with distanceFromRoot: 0
2016-05-17 20:45:49,028 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:49,028 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:49,029 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F8590CB2C4D643849371443AE7B4080B
2016-05-17 20:45:49,029 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,029 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:49,029 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517948800_0001_1_00, startRequestedTime=1463517949029, startedTime=1463517949029
2016-05-17 20:45:49,030 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:49,030 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:49,030 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:TASK_STARTED]: vertexName=F8590CB2C4D643849371443AE7B4080B, taskId=task_1463517948800_0001_1_00_000000, scheduledTime=1463517949030, launchTime=1463517949030
2016-05-17 20:45:49,030 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948800_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:49,031 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,031 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: F8590CB2C4D643849371443AE7B4080B, VertexParallelism: 1, TaskAttemptID:attempt_1463517948800_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=70DA86223CD7470C90C12B19E2874E00, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D03628992BF34FA1A5488F67378A92C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:49,031 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948800_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:49,031 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:49,035 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517948800_0001, with dag id: dag_1463517948800_0001_1
2016-05-17 20:45:49,038 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:49,040 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:49,040 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:49,041 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517948800_0001_00_000001
2016-05-17 20:45:49,041 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:49,043 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517948800_0001_00_000001
2016-05-17 20:45:49,043 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:49,045 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:49,045 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:49,046 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517948800_0001_00_000001, launchTime=1463517949045
2016-05-17 20:45:49,047 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517948800_0001_00_000001
2016-05-17 20:45:49,048 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517948800_0001_1_00_000000_0] to container: [container_1463517948800_0001_00_000001]
2016-05-17 20:45:49,049 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:49,049 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517948800_0001_00_000001 given task: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,049 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517948800_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,049 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517948800_0001_1_00_000000_0] started. Is using containerId: [container_1463517948800_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:49,049 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F8590CB2C4D643849371443AE7B4080B, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0, startTime=1463517949049, containerId=container_1463517948800_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517948800_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:49,050 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948800_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:49,050 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948800_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:49,050 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:49 Starting to run new task attempt: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,051 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,051 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,051 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:49,051 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:49,051 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:49,052 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:49,052 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:49,052 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: F8590CB2C4D643849371443AE7B4080B, VertexParallelism: 1, TaskAttemptID:attempt_1463517948800_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=70DA86223CD7470C90C12B19E2874E00, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D03628992BF34FA1A5488F67378A92C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:49,052 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:49,053 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,056 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:49,056 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,056 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517948800_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:49,058 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=70DA86223CD7470C90C12B19E2874E00, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:49,058 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,058 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D03628992BF34FA1A5488F67378A92C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:49,060 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:49,061 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:49,061 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:49,061 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,061 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D03628992BF34FA1A5488F67378A92C5
2016-05-17 20:45:49,061 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:49,062 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D03628992BF34FA1A5488F67378A92C5
2016-05-17 20:45:49,061 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,066 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:49,066 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,067 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,074 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F8590CB2C4D643849371443AE7B4080B, ordinal: 0
2016-05-17 20:45:49,074 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8590CB2C4D643849371443AE7B4080B, mem on start (mb), free: 159, total: 461, max: 672
2016-05-17 20:45:49,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:49,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:49,076 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:49,077 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D03628992BF34FA1A5488F67378A92C5
2016-05-17 20:45:49,077 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:49,077 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:49,077 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:49,077 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:49,077 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:49,078 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:49,078 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:49,078 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:49,078 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D03628992BF34FA1A5488F67378A92C5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:49,078 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 70DA86223CD7470C90C12B19E2874E00, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:49,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:49,079 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,079 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,079 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,079 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 70DA86223CD7470C90C12B19E2874E00 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:49,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:49,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:49,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:49,081 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,081 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:49,083 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1f6ed6d3
2016-05-17 20:45:49,084 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 20:45:49,084 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1f6ed6d3
2016-05-17 20:45:49,085 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:49,085 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:49,086 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,086 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"], id: D03628992BF34FA1A5488F67378A92C5
2016-05-17 20:45:49,087 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-17 20:45:49,087 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 20:45:49,088 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,088 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F8590CB2C4D643849371443AE7B4080B, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:49,091 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:49,092 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:49,092 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,093 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517948800_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:49,094 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,094 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8590CB2C4D643849371443AE7B4080B, mem on close (mb), free: 158, total: 461, max: 672
2016-05-17 20:45:49 Completed running task attempt: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,096 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:49,099 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:49,099 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F8590CB2C4D643849371443AE7B4080B, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0, startTime=1463517949049, finishTime=1463517949099, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 20:45:49,099 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948800_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:49,100 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,100 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:49,101 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:TASK_FINISHED]: vertexName=F8590CB2C4D643849371443AE7B4080B, taskId=task_1463517948800_0001_1_00_000000, startTime=1463517949049, finishTime=1463517949100, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463517948800_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 20:45:49,101 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948800_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:49,101 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] : 1
2016-05-17 20:45:49,101 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,101 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:49,102 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:49,103 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:VERTEX_FINISHED]: vertexName=F8590CB2C4D643849371443AE7B4080B, vertexId=vertex_1463517948800_0001_1_00, initRequestedTime=1463517949007, initedTime=1463517949028, startRequestedTime=1463517949029, startedTime=1463517949029, finishTime=1463517949101, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1463517949049, firstTasksToStart=[ task_1463517948800_0001_1_00_000000 ], lastTaskFinishTime=1463517949099, lastTasksToFinish=[ task_1463517948800_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517948800_0001_1_00_000000 ], longestDurationTasks=[ task_1463517948800_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:49,103 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:49,104 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-05-17 20:45:49,104 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:49,104 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:49,105 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:49,105 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517948800_0001_00_000001, stoppedTime=1463517949105, exitStatus=0
2016-05-17 20:45:49,105 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,105 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:49,105 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:49,106 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517948800_0001_1
2016-05-17 20:45:49,106 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517948800_0001_1
2016-05-17 20:45:49,106 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D03628992BF34FA1A5488F67378A92C5 for vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,106 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517948800_0001_00_000001
2016-05-17 20:45:49,107 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517948800_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:49,107 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517948800_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:49,108 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517948800_0001_00_000001, Exiting
2016-05-17 20:45:49,108 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517948800_0001_00_000001
2016-05-17 20:45:49,108 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517948800_0001_00_000001
2016-05-17 20:45:49,112 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517948800_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:49,112 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517948800_0001_00_000001 completed successfully
2016-05-17 20:45:49,118 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517948800_0001_1, queueSize=0
2016-05-17 20:45:49,119 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:49,119 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517948800_0001_1, startTime=1463517949007, finishTime=1463517949117, timeTaken=110, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 20:45:49,119 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517948800_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:49,119 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948800_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:49,119 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517948800_0001_00_000001
2016-05-17 20:45:49,119 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:49 Completed Dag: dag_1463517948800_0001_1
2016-05-17 20:45:49,120 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517948800_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:49,120 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517948800_0001_1_post
2016-05-17 20:45:49,120 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,120 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:49,133 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463517948800_0001
2016-05-17 20:45:49,136 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:49,136 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:49,136 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:49,136 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:49,137 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:49,137 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:49,137 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:49,138 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:49,138 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:49,138 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:49,139 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:49,139 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:49,140 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:49,144 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001
2016-05-17 20:45:49,145 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped/_temporary
2016-05-17 20:45:49,146 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.369
2016-05-17 20:45:49,182 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-17 20:45:49,213 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:49,213 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:49,271 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-17 20:45:49,286 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-05-17 20:45:49,287 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:49,287 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:49,288 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:49,306 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-05-17 20:45:49,320 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 20:45:49,328 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-17 20:45:49,427 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-17 20:45:49,428 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:49,429 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:49,430 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-17 20:45:49,430 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  parallel execution of steps is enabled: false
2016-05-17 20:45:49,430 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  executing total steps: 1
2016-05-17 20:45:49,430 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  allocating management threads: 1
2016-05-17 20:45:49,431 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49,431 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:49,447 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-17 20:45:49,452 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:49,453 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:49,453 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:49,463 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001 doesn't exist and is created
2016-05-17 20:45:49,480 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:49,480 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:49,481 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001
2016-05-17 20:45:49,486 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517949453_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:49,498 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:49,499 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:49,502 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:49,502 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:49,502 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001/localmode-log-dir
2016-05-17 20:45:49,502 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001/localmode-log-dir/history.txt.appattempt_1463517949453_0001_000000, maxErrors=10
2016-05-17 20:45:49,502 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:49,503 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517949453_0001_000000, appSubmitTime=1463517949485, launchTime=1463517949486
2016-05-17 20:45:49,508 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:49,509 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:49,511 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:49,511 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:49,513 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50928
2016-05-17 20:45:49,515 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:49,517 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517949453_0001_000000, startTime=1463517949517
2016-05-17 20:45:49,517 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:49,517 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:49,580 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:49,580 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:49,580 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463517949453_0001, dagName=[5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49,598 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49,600 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517949453_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001/localmode-log-dir/dag_1463517949453_0001_1.dot
2016-05-17 20:45:49,602 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001/localmode-log-dir/dag_1463517949453_0001_1-tez-dag.pb.txt
2016-05-17 20:45:49,630 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517949453_0001_1
2016-05-17 20:45:49,630 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,631 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49 Running Dag: dag_1463517949453_0001_1
2016-05-17 20:45:49,643 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517949453_0001_1, submitTime=1463517949598
2016-05-17 20:45:49,643 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:49,644 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,646 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:49,647 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:49,647 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517949453_0001_1, initTime=1463517949643
2016-05-17 20:45:49,647 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517949453_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:49,648 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_STARTED]: dagID=dag_1463517949453_0001_1, startTime=1463517949648
2016-05-17 20:45:49,648 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:49,649 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:49,649 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517949453_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:49,649 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D5951A42F5F949D9AD7CF3972EB9E22E : {F7A74F1017004C2E896FE02CA641E7F9={InputName=F7A74F1017004C2E896FE02CA641E7F9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:49,649 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F7A74F1017004C2E896FE02CA641E7F9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:49,649 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,650 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,650 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,650 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]. Starting root input initializers: 1
2016-05-17 20:45:49,651 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:49,651 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CDB30EBFD3D44B38AFF1F4B60286850E : {A154428B660F4B11BBAB47BE399BB520={InputName=A154428B660F4B11BBAB47BE399BB520}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:49,651 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A154428B660F4B11BBAB47BE399BB520, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:49,651 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,651 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,651 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,652 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]. Starting root input initializers: 1
2016-05-17 20:45:49,652 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F7A74F1017004C2E896FE02CA641E7F9 on vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,653 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:49,654 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F7A74F1017004C2E896FE02CA641E7F9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:49,654 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:49,654 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:49,656 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:49,656 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:49,657 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,657 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F7A74F1017004C2E896FE02CA641E7F9 on vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,657 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463517949453_0001, dagName=[5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49,660 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:49,660 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,660 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,660 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,660 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A154428B660F4B11BBAB47BE399BB520 on vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,660 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4A4B6074FD794C7F807C3AC2D233A01D, vertexId=vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:49,662 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4A4B6074FD794C7F807C3AC2D233A01D, vertexId=vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,664 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4A4B6074FD794C7F807C3AC2D233A01D, vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:49,664 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4A4B6074FD794C7F807C3AC2D233A01D, vertexId=vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,666 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:49,666 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:49,666 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A154428B660F4B11BBAB47BE399BB520 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:49,667 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, vertexId=vertex_1463517949453_0001_1_02, initRequestedTime=1463517949657, initedTime=1463517949666, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:49,667 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:49,667 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,668 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F7A74F1017004C2E896FE02CA641E7F9=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:49,668 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] parallelism set to 1
2016-05-17 20:45:49,669 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,670 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517949453_0001_1_00, eventCount=1
2016-05-17 20:45:49,671 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:49,671 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, vertexId=vertex_1463517949453_0001_1_00, initRequestedTime=1463517949649, initedTime=1463517949671, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:49,672 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] with distanceFromRoot: 0
2016-05-17 20:45:49,672 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:49,672 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:49,673 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,673 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:49,673 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517949453_0001_1_00, startRequestedTime=1463517949672, startedTime=1463517949672
2016-05-17 20:45:49,674 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:49,674 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:49,671 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:49,675 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:49,675 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_STARTED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, taskId=task_1463517949453_0001_1_00_000000, scheduledTime=1463517949674, launchTime=1463517949674
2016-05-17 20:45:49,676 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:49,676 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517949453_0001_1_00 for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] numStartedSources: 1 numSources: 2
2016-05-17 20:45:49,676 INFO  impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] numStartedSources: 1 numSources: 2
2016-05-17 20:45:49,677 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,677 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: D5951A42F5F949D9AD7CF3972EB9E22E, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7A74F1017004C2E896FE02CA641E7F9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:49,678 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:49,678 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:49,679 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:49,677 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:49,680 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:49,682 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:49,683 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517949453_0001_00_000001
2016-05-17 20:45:49,683 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:49,684 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A154428B660F4B11BBAB47BE399BB520 on vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,685 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,685 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A154428B660F4B11BBAB47BE399BB520=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:49,685 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517949453_0001_00_000001
2016-05-17 20:45:49,685 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] parallelism set to 1
2016-05-17 20:45:49,685 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:49,686 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,686 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517949453_0001_1_01, eventCount=1
2016-05-17 20:45:49,689 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, vertexId=vertex_1463517949453_0001_1_01, initRequestedTime=1463517949651, initedTime=1463517949689, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:49,689 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] with distanceFromRoot: 0
2016-05-17 20:45:49,689 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:49,689 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:49,689 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,689 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517949453_0001_00_000001, launchTime=1463517949689
2016-05-17 20:45:49,690 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:49,690 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:49,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517949453_0001_1_01, startRequestedTime=1463517949689, startedTime=1463517949689
2016-05-17 20:45:49,690 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:49,690 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:49,691 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:49,691 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000001
2016-05-17 20:45:49,691 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:49,691 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_STARTED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, taskId=task_1463517949453_0001_1_01_000000, scheduledTime=1463517949691, launchTime=1463517949691
2016-05-17 20:45:49,694 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:49,694 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463517949453_0001, with dag id: dag_1463517949453_0001_1
2016-05-17 20:45:49,692 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517949453_0001_1_00_000000_0] to container: [container_1463517949453_0001_00_000001]
2016-05-17 20:45:49,694 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517949453_0001_1_01 for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] numStartedSources: 2 numSources: 2
2016-05-17 20:45:49,695 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:49,695 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] with distanceFromRoot: 1
2016-05-17 20:45:49,695 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517949453_0001_00_000001 given task: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,695 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,695 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,697 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517949453_0001_1_00_000000_0] started. Is using containerId: [container_1463517949453_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:49 Starting to run new task attempt: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,697 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,697 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0, startTime=1463517949696, containerId=container_1463517949453_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517949453_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:49,697 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:49,697 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:49,698 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CDB30EBFD3D44B38AFF1F4B60286850E in vertex: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,698 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,698 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D5951A42F5F949D9AD7CF3972EB9E22E in vertex: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,698 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:49,698 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CFD26EBBC047461FA4A780DCF2EDE82F with 2 source tasks and 1 pending tasks
2016-05-17 20:45:49,698 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:49,699 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517949453_0001_1_02, startRequestedTime=1463517949697, startedTime=1463517949697
2016-05-17 20:45:49,699 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:49,699 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:49,699 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:49,699 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:49,699 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: D5951A42F5F949D9AD7CF3972EB9E22E, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7A74F1017004C2E896FE02CA641E7F9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:49,700 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:49,699 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:49,707 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,707 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:49,707 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,708 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F7A74F1017004C2E896FE02CA641E7F9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:49,708 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,710 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:49,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:49,711 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,713 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:49,714 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:49,714 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,720 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,721 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517949453_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:49,720 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: CDB30EBFD3D44B38AFF1F4B60286850E, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A154428B660F4B11BBAB47BE399BB520, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:49,721 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:49,722 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:49,722 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:49,720 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:49,727 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,728 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,728 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517949453_0001_00_000002
2016-05-17 20:45:49,728 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:49,732 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517949453_0001_00_000002
2016-05-17 20:45:49,732 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:49,737 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5951A42F5F949D9AD7CF3972EB9E22E, ordinal: 1
2016-05-17 20:45:49,737 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5951A42F5F949D9AD7CF3972EB9E22E, mem on start (mb), free: 46, total: 461, max: 672
2016-05-17 20:45:49,737 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,738 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:49,738 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:49,743 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:49,743 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,743 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,744 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:49,752 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:49,752 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:49,753 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517949453_0001_00_000002, launchTime=1463517949752
2016-05-17 20:45:49,754 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:49,755 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:49,755 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:49,756 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:49,756 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:49,757 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:49,757 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CFD26EBBC047461FA4A780DCF2EDE82F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:49,758 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F7A74F1017004C2E896FE02CA641E7F9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:49,758 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:49,759 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,759 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,759 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,759 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000002
2016-05-17 20:45:49,761 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F7A74F1017004C2E896FE02CA641E7F9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:49,762 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:49,762 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:49,764 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:49,763 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517949453_0001_1_01_000000_0] to container: [container_1463517949453_0001_00_000002]
2016-05-17 20:45:49,764 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:49,765 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517949453_0001_1_01_000000_0] started. Is using containerId: [container_1463517949453_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:49,765 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0, startTime=1463517949765, containerId=container_1463517949453_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517949453_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:49,766 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:49,766 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:49,766 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517949453_0001_00_000002 given task: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,766 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,766 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:49 Starting to run new task attempt: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,767 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,768 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,768 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:49,768 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:49,768 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:49,768 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:49,768 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,768 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:49,772 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: CDB30EBFD3D44B38AFF1F4B60286850E, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A154428B660F4B11BBAB47BE399BB520, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:49,772 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:49,772 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:49,776 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,776 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:49,776 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:49,778 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:49,778 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,779 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:49,779 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,779 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2c32d8b8
2016-05-17 20:45:49,786 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A154428B660F4B11BBAB47BE399BB520, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:49,787 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:49,792 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:49,793 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517949453_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:49,786 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1dddf775; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@57a131a4; comparator=cascading.tuple.hadoop.util.TupleComparator@536fe809; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:49,793 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:49,794 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:49,794 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:49,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:49,831 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,792 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,831 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,833 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:49,833 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,787 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,787 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2c32d8b8
2016-05-17 20:45:49,834 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:49,834 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:49,844 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:49,844 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:49,845 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,844 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,845 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:49,845 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:49,846 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:49,846 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:49,846 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:49,846 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:49,847 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,847 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5951A42F5F949D9AD7CF3972EB9E22E, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:49,852 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,852 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5951A42F5F949D9AD7CF3972EB9E22E, mem on close (mb), free: 136, total: 439, max: 672
2016-05-17 20:45:49,852 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:49,853 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:49,853 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:49,853 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:49,858 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:49,858 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CDB30EBFD3D44B38AFF1F4B60286850E, ordinal: 0
2016-05-17 20:45:49,859 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDB30EBFD3D44B38AFF1F4B60286850E, mem on start (mb), free: 135, total: 439, max: 672
2016-05-17 20:45:49,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:49,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:49,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:49,860 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:49,860 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:49,860 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:49,860 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:49,861 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:49,861 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:49,861 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A154428B660F4B11BBAB47BE399BB520, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:49,861 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CFD26EBBC047461FA4A780DCF2EDE82F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:49,861 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:49,861 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,862 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A154428B660F4B11BBAB47BE399BB520 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:49,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:49,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:49,863 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:49,863 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49 Completed running task attempt: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,867 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:49,867 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:49,867 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,868 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:49,869 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517949453_0001_1_00, eventCount=1
2016-05-17 20:45:49,869 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:49,869 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517949847, Process_Duration=5, Process_End_Time=1463517949852, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:49,869 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:49,870 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0, startTime=1463517949696, finishTime=1463517949869, timeTaken=173, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949847, Process_Duration=5, Process_End_Time=1463517949852, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:49,870 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:49,871 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,871 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:49,871 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:49,872 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_FINISHED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, taskId=task_1463517949453_0001_1_00_000000, startTime=1463517949696, finishTime=1463517949871, timeTaken=175, status=SUCCEEDED, successfulAttemptID=attempt_1463517949453_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949847, Process_Duration=5, Process_End_Time=1463517949852, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:49,872 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:49,872 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] : 1
2016-05-17 20:45:49,872 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,873 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:49,873 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@424c8abf; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@2f29239a; comparator=cascading.tuple.hadoop.util.TupleComparator@35e06ba6; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:49,874 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_FINISHED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, vertexId=vertex_1463517949453_0001_1_00, initRequestedTime=1463517949649, initedTime=1463517949671, startRequestedTime=1463517949672, startedTime=1463517949672, finishTime=1463517949872, timeTaken=200, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949847, Process_Duration=5, Process_End_Time=1463517949852, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517949696, firstTasksToStart=[ task_1463517949453_0001_1_00_000000 ], lastTaskFinishTime=1463517949869, lastTasksToFinish=[ task_1463517949453_0001_1_00_000000 ], minTaskDuration=173, maxTaskDuration=173, avgTaskDuration=173.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517949453_0001_1_00_000000 ], longestDurationTasks=[ task_1463517949453_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:49,875 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:49,875 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:49,874 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:49,875 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] attempt: attempt_1463517949453_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:49,876 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517949453_0001_00_000001, stoppedTime=1463517949876, exitStatus=0
2016-05-17 20:45:49,876 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:49,876 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:49,876 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:49,876 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:49,877 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:49,877 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:49,873 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:49,873 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,876 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517949453_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:49,882 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:49,883 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2cf1567a
2016-05-17 20:45:49,883 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000001
2016-05-17 20:45:49,885 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517949453_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:49,885 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:49,885 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517949453_0001_00_000001, Exiting
2016-05-17 20:45:49,885 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517949453_0001_00_000001
2016-05-17 20:45:49,886 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517949453_0001_00_000001
2016-05-17 20:45:49,885 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2cf1567a
2016-05-17 20:45:49,895 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:49,896 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:49,895 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517949453_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:49,894 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:49,896 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:49,897 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:49,897 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:49,897 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:49,897 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517949453_0001_00_000001 completed successfully
2016-05-17 20:45:49,897 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:49,900 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:49,900 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,900 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CDB30EBFD3D44B38AFF1F4B60286850E, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:49,904 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,905 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDB30EBFD3D44B38AFF1F4B60286850E, mem on close (mb), free: 31, total: 439, max: 672
2016-05-17 20:45:49,905 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:49,906 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:49,906 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:49,906 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:49,910 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:49 Completed running task attempt: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,916 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:49,917 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517949453_0001_1_01, eventCount=1
2016-05-17 20:45:49,917 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517949900, Process_Duration=4, Process_End_Time=1463517949904, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:49,918 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0, startTime=1463517949765, finishTime=1463517949918, timeTaken=153, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949900, Process_Duration=4, Process_End_Time=1463517949904, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:49,918 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:49,919 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,920 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_FINISHED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, taskId=task_1463517949453_0001_1_01_000000, startTime=1463517949765, finishTime=1463517949919, timeTaken=154, status=SUCCEEDED, successfulAttemptID=attempt_1463517949453_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949900, Process_Duration=4, Process_End_Time=1463517949904, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:49,920 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:49,920 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] : 1
2016-05-17 20:45:49,920 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,920 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:49,918 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:49,922 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:49,922 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_FINISHED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, vertexId=vertex_1463517949453_0001_1_01, initRequestedTime=1463517949651, initedTime=1463517949689, startRequestedTime=1463517949689, startedTime=1463517949689, finishTime=1463517949920, timeTaken=231, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949900, Process_Duration=4, Process_End_Time=1463517949904, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517949765, firstTasksToStart=[ task_1463517949453_0001_1_01_000000 ], lastTaskFinishTime=1463517949918, lastTasksToFinish=[ task_1463517949453_0001_1_01_000000 ], minTaskDuration=153, maxTaskDuration=153, avgTaskDuration=153.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517949453_0001_1_01_000000 ], longestDurationTasks=[ task_1463517949453_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:49,922 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:49,922 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] attempt: attempt_1463517949453_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:49,922 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,922 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,922 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:49,923 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:49,923 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517949453_0001_00_000002, stoppedTime=1463517949923, exitStatus=0
2016-05-17 20:45:49,923 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:49,923 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:49,924 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:49,924 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_STARTED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, taskId=task_1463517949453_0001_1_02_000000, scheduledTime=1463517949924, launchTime=1463517949924
2016-05-17 20:45:49,924 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:49,924 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:49,924 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,924 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:49,925 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,925 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517949453_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 20:45:49,925 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: CFD26EBBC047461FA4A780DCF2EDE82F, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5951A42F5F949D9AD7CF3972EB9E22E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=CDB30EBFD3D44B38AFF1F4B60286850E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4A4B6074FD794C7F807C3AC2D233A01D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:49,926 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:49,926 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:49,926 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:49,926 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000002
2016-05-17 20:45:50,128 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517949453_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:50,128 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:50,130 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517949453_0001_00_000002, Exiting
2016-05-17 20:45:50,130 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517949453_0001_00_000002
2016-05-17 20:45:50,130 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517949453_0001_00_000002
2016-05-17 20:45:50,132 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517949453_0001_00_000003
2016-05-17 20:45:50,132 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:50,132 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517949453_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:50,132 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517949453_0001_00_000002 completed successfully
2016-05-17 20:45:50,132 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:50,134 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517949453_0001_00_000003
2016-05-17 20:45:50,135 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:50,142 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:50,142 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:50,142 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517949453_0001_00_000003, launchTime=1463517950142
2016-05-17 20:45:50,143 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000003
2016-05-17 20:45:50,144 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517949453_0001_1_02_000000_0] to container: [container_1463517949453_0001_00_000003]
2016-05-17 20:45:50,144 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:50,145 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517949453_0001_00_000003 given task: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,145 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517949453_0001_1_02_000000_0] started. Is using containerId: [container_1463517949453_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 20:45:50,145 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0, startTime=1463517950145, containerId=container_1463517949453_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517949453_0001_00_000003/teamcity, completedLogs=
2016-05-17 20:45:50,145 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:50,146 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:50,145 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,145 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:50 Starting to run new task attempt: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,147 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,147 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,148 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:50,148 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:50,148 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:50,148 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:50,148 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:50,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: CFD26EBBC047461FA4A780DCF2EDE82F, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5951A42F5F949D9AD7CF3972EB9E22E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=CDB30EBFD3D44B38AFF1F4B60286850E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4A4B6074FD794C7F807C3AC2D233A01D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:50,148 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:50,152 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:50,153 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,152 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517949453_0001_1_02_000000_0 sent events: (0-2)
2016-05-17 20:45:50,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D5951A42F5F949D9AD7CF3972EB9E22E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:50,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:50,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,155 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CDB30EBFD3D44B38AFF1F4B60286850E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:50,156 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:50,156 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,158 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4A4B6074FD794C7F807C3AC2D233A01D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:50,158 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:50,158 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:50,159 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:50,159 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,160 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,159 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,159 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:50,161 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4A4B6074FD794C7F807C3AC2D233A01D
2016-05-17 20:45:50,162 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:50,162 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4A4B6074FD794C7F807C3AC2D233A01D
2016-05-17 20:45:50,166 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:50,167 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:50,167 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,173 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CFD26EBBC047461FA4A780DCF2EDE82F, ordinal: 2
2016-05-17 20:45:50,173 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CFD26EBBC047461FA4A780DCF2EDE82F, mem on start (mb), free: 311, total: 333, max: 672
2016-05-17 20:45:50,173 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,174 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-05-17 20:45:50,174 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:50,175 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:50,178 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:50,178 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4A4B6074FD794C7F807C3AC2D233A01D
2016-05-17 20:45:50,178 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:50,178 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:50,179 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:50,180 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:50,180 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:50,180 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:50,180 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-17 20:45:50,181 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-05-17 20:45:50,181 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CDB30EBFD3D44B38AFF1F4B60286850E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 20:45:50,181 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4A4B6074FD794C7F807C3AC2D233A01D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:50,181 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D5951A42F5F949D9AD7CF3972EB9E22E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 20:45:50,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:50,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,183 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:50,183 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:50,184 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D5951A42F5F949D9AD7CF3972EB9E22E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:50,184 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 20:45:50,184 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:50,184 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:50,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D5951A42F5F949D9AD7CF3972EB9E22E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:50,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CDB30EBFD3D44B38AFF1F4B60286850E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:50,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-05-17 20:45:50,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-05-17 20:45:50,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,187 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,188 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:50,189 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:50,189 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:50,189 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CDB30EBFD3D44B38AFF1F4B60286850E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:50,189 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 20:45:50,189 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:50,189 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:50,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,191 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:50,196 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,201 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]]
2016-05-17 20:45:50,202 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] streamed: true, id: 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:50,202 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"], id: 4A4B6074FD794C7F807C3AC2D233A01D
2016-05-17 20:45:50,202 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-17 20:45:50,202 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 5F8FD08921AD47D1B0A054A29A12F241, for 2 inputs
2016-05-17 20:45:50,203 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 5F8FD08921AD47D1B0A054A29A12F241, for 2 inputs
2016-05-17 20:45:50,211 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517949453_0001_1_00_000000_0_10023, runDuration: 152804, ]
2016-05-17 20:45:50,213 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517949453_0001_1_00_000000_0_10023] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517950213, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 20:45:50,213 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,213 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:50,213 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D5951A42F5F949D9AD7CF3972EB9E22E] #4 in 1ms
2016-05-17 20:45:50,214 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D5951A42F5F949D9AD7CF3972EB9E22E] #4, status:false, isInterrupted:false
2016-05-17 20:45:50,214 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517949453_0001_1_01_000000_0_10027, runDuration: 82522, ]
2016-05-17 20:45:50,215 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517949453_0001_1_01_000000_0_10027] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517950215, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 20:45:50,216 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,216 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:50,216 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CDB30EBFD3D44B38AFF1F4B60286850E] #5 in 2ms
2016-05-17 20:45:50,216 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:50,217 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:50,218 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:50,218 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:50,218 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:50,218 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,219 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CDB30EBFD3D44B38AFF1F4B60286850E] #5, status:false, isInterrupted:false
2016-05-17 20:45:50,219 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:50,220 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:50,220 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 20:45:50,220 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 20:45:50,221 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:50,221 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:50,221 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:50,221 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:50,222 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,222 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:50,226 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CFD26EBBC047461FA4A780DCF2EDE82F, all 2 inputs ready in: 00:00:00.000
2016-05-17 20:45:50,228 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@904c609; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@18657d7f
2016-05-17 20:45:50,230 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@2f12295c; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@51596f28
2016-05-17 20:45:50,234 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:50,235 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:50,235 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,237 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517949453_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 20:45:50,238 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,238 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CFD26EBBC047461FA4A780DCF2EDE82F, mem on close (mb), free: 305, total: 333, max: 672
2016-05-17 20:45:50,238 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,239 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:50,240 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:50,240 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:50,241 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:50,241 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:50,241 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,242 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:50,243 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:50,244 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:50,244 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:50,244 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:50 Completed running task attempt: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,246 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:50,247 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517950226, Process_Duration=12, Process_End_Time=1463517950238, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 20:45:50,247 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:50,248 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0, startTime=1463517950145, finishTime=1463517950247, timeTaken=102, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950226, Process_Duration=12, Process_End_Time=1463517950238, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:50,248 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:50,248 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:50,248 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,249 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:50,250 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_FINISHED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, taskId=task_1463517949453_0001_1_02_000000, startTime=1463517950145, finishTime=1463517950249, timeTaken=104, status=SUCCEEDED, successfulAttemptID=attempt_1463517949453_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950226, Process_Duration=12, Process_End_Time=1463517950238, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:50,250 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:50,250 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] : 1
2016-05-17 20:45:50,250 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:50,251 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:50,251 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,252 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000003
2016-05-17 20:45:50,252 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:50,253 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_FINISHED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, vertexId=vertex_1463517949453_0001_1_02, initRequestedTime=1463517949657, initedTime=1463517949666, startRequestedTime=1463517949697, startedTime=1463517949697, finishTime=1463517950250, timeTaken=553, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950226, Process_Duration=12, Process_End_Time=1463517950238, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463517950145, firstTasksToStart=[ task_1463517949453_0001_1_02_000000 ], lastTaskFinishTime=1463517950247, lastTasksToFinish=[ task_1463517949453_0001_1_02_000000 ], minTaskDuration=102, maxTaskDuration=102, avgTaskDuration=102.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517949453_0001_1_02_000000 ], longestDurationTasks=[ task_1463517949453_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:50,253 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:50,253 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:50,253 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517949453_0001_00_000003, stoppedTime=1463517950253, exitStatus=0
2016-05-17 20:45:50,253 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:50,254 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:50,254 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:50,254 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517949453_0001_1
2016-05-17 20:45:50,254 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517949453_0001_1
2016-05-17 20:45:50,255 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:50,255 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:50,255 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 4A4B6074FD794C7F807C3AC2D233A01D for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:50,256 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517949453_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 20:45:50,256 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:50,256 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517949453_0001_00_000003, Exiting
2016-05-17 20:45:50,256 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517949453_0001_00_000003
2016-05-17 20:45:50,256 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517949453_0001_00_000003
2016-05-17 20:45:50,263 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517949453_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:50,263 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517949453_0001_00_000003 completed successfully
2016-05-17 20:45:50,273 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517949453_0001_1, queueSize=0
2016-05-17 20:45:50,274 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:50,278 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517949453_0001_1, startTime=1463517949648, finishTime=1463517950270, timeTaken=622, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=440, FILE_BYTES_WRITTEN=8334, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=70, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1270349824, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390553849973, Process_Duration=21, Process_End_Time=4390553849994, Read_Duration=8, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-17 20:45:50,279 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517949453_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:50,279 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517949453_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:50,280 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517949453_0001_00_000003
2016-05-17 20:45:50,280 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:50 Completed Dag: dag_1463517949453_0001_1
2016-05-17 20:45:50,282 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517949453_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:50,283 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517949453_0001_1_post
2016-05-17 20:45:50,283 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,283 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:50,288 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463517949453_0001
2016-05-17 20:45:50,290 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:50,290 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:50,290 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:50,290 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:50,290 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:50,291 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:50,292 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:50,292 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:50,292 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:50,292 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:50,293 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:50,293 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:50,293 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:50,298 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001
2016-05-17 20:45:50,299 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown/_temporary
2016-05-17 20:45:50,300 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  completed in: 00:00.872
2016-05-17 20:45:50,374 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-17 20:45:50,398 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:50,398 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:50,446 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-17 20:45:50,462 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-05-17 20:45:50,462 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:50,462 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:50,463 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:50,481 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:50,490 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-17 20:45:50,542 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:50,542 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:50,542 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-17 20:45:50,543 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:50,543 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:50,543 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:50,544 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...leRelative/simplerelative
2016-05-17 20:45:50,546 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:50,559 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:50,563 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:50,564 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:50,564 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:50,573 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001 doesn't exist and is created
2016-05-17 20:45:50,589 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:50,590 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:50,590 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001
2016-05-17 20:45:50,593 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517950564_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:50,604 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:50,604 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:50,606 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:50,606 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:50,607 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001/localmode-log-dir
2016-05-17 20:45:50,607 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001/localmode-log-dir/history.txt.appattempt_1463517950564_0001_000000, maxErrors=10
2016-05-17 20:45:50,607 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:50,607 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517950564_0001_000000, appSubmitTime=1463517950593, launchTime=1463517950593
2016-05-17 20:45:50,610 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:50,612 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:50,616 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:50,616 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:50,616 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49911
2016-05-17 20:45:50,618 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:50,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517950564_0001_000000, startTime=1463517950620
2016-05-17 20:45:50,620 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:50,620 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:50,690 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:50,690 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:50,690 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463517950564_0001, dagName=[BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative
2016-05-17 20:45:50,707 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative
2016-05-17 20:45:50,709 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517950564_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001/localmode-log-dir/dag_1463517950564_0001_1.dot
2016-05-17 20:45:50,710 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001/localmode-log-dir/dag_1463517950564_0001_1-tez-dag.pb.txt
2016-05-17 20:45:50,743 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517950564_0001_1
2016-05-17 20:45:50,743 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,743 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative
2016-05-17 20:45:50 Running Dag: dag_1463517950564_0001_1
2016-05-17 20:45:50,763 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517950564_0001_1, submitTime=1463517950708
2016-05-17 20:45:50,763 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:50,765 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,765 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:50,765 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:50,766 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517950564_0001_1, initTime=1463517950764
2016-05-17 20:45:50,766 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517950564_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:50,766 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_STARTED]: dagID=dag_1463517950564_0001_1, startTime=1463517950766
2016-05-17 20:45:50,766 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:50,767 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517950564_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:50,767 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B0968C4BAD2B45BF940D458AF443ECFD : {5C0FB077FC594C67B8A6CB9357BFFD92={InputName=5C0FB077FC594C67B8A6CB9357BFFD92}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:50,767 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5C0FB077FC594C67B8A6CB9357BFFD92, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:50,767 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,767 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,767 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,768 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]. Starting root input initializers: 1
2016-05-17 20:45:50,768 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:50,768 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:50,769 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,772 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:50,772 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,774 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5C0FB077FC594C67B8A6CB9357BFFD92 on vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,774 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,775 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,774 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463517950564_0001, dagName=[BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative
2016-05-17 20:45:50,776 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=09D0E48849B241658D0742457F342690, vertexId=vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:50,776 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=09D0E48849B241658D0742457F342690, vertexId=vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,780 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=09D0E48849B241658D0742457F342690, vertexName=FCF45131816E4FC5947C4DDFA51C1448, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:50,781 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=09D0E48849B241658D0742457F342690, vertexId=vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,782 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:50,782 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5C0FB077FC594C67B8A6CB9357BFFD92 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:50,784 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:50,784 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:50,785 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:50,784 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, vertexId=vertex_1463517950564_0001_1_01, initRequestedTime=1463517950769, initedTime=1463517950784, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:50,786 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:50,788 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:50,792 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5C0FB077FC594C67B8A6CB9357BFFD92 on vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,792 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:50,793 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5C0FB077FC594C67B8A6CB9357BFFD92=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:50,793 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] parallelism set to 1
2016-05-17 20:45:50,794 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:50,794 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517950564_0001_1_00, eventCount=1
2016-05-17 20:45:50,802 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, vertexId=vertex_1463517950564_0001_1_00, initRequestedTime=1463517950767, initedTime=1463517950802, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:50,802 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:50,802 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] with distanceFromRoot: 0
2016-05-17 20:45:50,803 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:50,803 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:50,803 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,804 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:50,804 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517950564_0001_1_00, startRequestedTime=1463517950803, startedTime=1463517950803
2016-05-17 20:45:50,805 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:50,805 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:50,805 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517950564_0001, with dag id: dag_1463517950564_0001_1
2016-05-17 20:45:50,810 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_STARTED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, taskId=task_1463517950564_0001_1_00_000000, scheduledTime=1463517950805, launchTime=1463517950805
2016-05-17 20:45:50,810 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:50,811 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517950564_0001_1_00 for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] numStartedSources: 1 numSources: 1
2016-05-17 20:45:50,811 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] with distanceFromRoot: 1
2016-05-17 20:45:50,811 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,811 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B0968C4BAD2B45BF940D458AF443ECFD in vertex: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,812 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FCF45131816E4FC5947C4DDFA51C1448 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:50,812 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:50,812 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517950564_0001_1_01, startRequestedTime=1463517950811, startedTime=1463517950811
2016-05-17 20:45:50,812 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:50,812 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative, VertexName: B0968C4BAD2B45BF940D458AF443ECFD, VertexParallelism: 1, TaskAttemptID:attempt_1463517950564_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5C0FB077FC594C67B8A6CB9357BFFD92, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FCF45131816E4FC5947C4DDFA51C1448, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:50,813 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:50,813 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:50,815 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:50,827 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:50,828 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:50,828 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517950564_0001_00_000001
2016-05-17 20:45:50,828 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:50,830 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517950564_0001_00_000001
2016-05-17 20:45:50,830 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:50,831 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:50,831 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:50,832 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517950564_0001_00_000001, launchTime=1463517950831
2016-05-17 20:45:50,833 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517950564_0001_00_000001
2016-05-17 20:45:50,834 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517950564_0001_1_00_000000_0] to container: [container_1463517950564_0001_00_000001]
2016-05-17 20:45:50,834 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:50,834 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517950564_0001_1_00_000000_0] started. Is using containerId: [container_1463517950564_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:50,835 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0, startTime=1463517950834, containerId=container_1463517950564_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517950564_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:50,835 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:50,835 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:50,836 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:50,836 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517950564_0001_00_000001 given task: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,836 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517950564_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50 Starting to run new task attempt: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,837 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,837 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,837 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:50,837 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:50,837 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:50,837 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:50,838 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:50,838 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative, VertexName: B0968C4BAD2B45BF940D458AF443ECFD, VertexParallelism: 1, TaskAttemptID:attempt_1463517950564_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5C0FB077FC594C67B8A6CB9357BFFD92, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FCF45131816E4FC5947C4DDFA51C1448, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:50,838 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:50,848 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,848 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:50,848 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,850 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5C0FB077FC594C67B8A6CB9357BFFD92, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:50,851 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:50,851 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,852 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,854 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,856 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517950564_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:50,861 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FCF45131816E4FC5947C4DDFA51C1448, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:50,861 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:50,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,863 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:50,865 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:50,866 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,867 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:50,867 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,876 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B0968C4BAD2B45BF940D458AF443ECFD, ordinal: 0
2016-05-17 20:45:50,876 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B0968C4BAD2B45BF940D458AF443ECFD, mem on start (mb), free: 213, total: 333, max: 672
2016-05-17 20:45:50,877 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,877 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:50,877 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:50,877 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:50,877 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:50,877 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:50,878 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:50,878 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:50,878 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:50,879 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:50,879 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5C0FB077FC594C67B8A6CB9357BFFD92, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:50,879 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FCF45131816E4FC5947C4DDFA51C1448, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:50,879 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:50,880 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,880 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:50,880 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,880 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5C0FB077FC594C67B8A6CB9357BFFD92 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:50,881 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:50,881 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:50,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:50,882 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,886 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:50,888 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,889 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 266B1E76DC3542AB9322E27C80F9F0EE
2016-05-17 20:45:50,889 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 266B1E76DC3542AB9322E27C80F9F0EE
2016-05-17 20:45:50,893 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:50,894 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1a2e7f83; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@62b57ee2; comparator=cascading.tuple.hadoop.util.TupleComparator@799d5a0e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:50,894 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@409335e1
2016-05-17 20:45:50,894 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:50,895 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:50,895 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:50,896 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@409335e1
2016-05-17 20:45:50,896 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:50,896 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:50,911 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:50,912 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:50,912 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:50,912 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:50,913 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:50,914 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:50,914 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:50,914 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B0968C4BAD2B45BF940D458AF443ECFD, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:50,918 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,919 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B0968C4BAD2B45BF940D458AF443ECFD, mem on close (mb), free: 111, total: 333, max: 672
2016-05-17 20:45:50,919 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:50,920 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:50,920 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 20:45:50,921 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 20:45:50,926 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:50 Completed running task attempt: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,930 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:50,931 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517950564_0001_1_00, eventCount=1
2016-05-17 20:45:50,932 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0, startTime=1463517950834, finishTime=1463517950932, timeTaken=98, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950914, Process_Duration=4, Process_End_Time=1463517950918, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:50,933 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:50,933 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:50,933 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,934 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_FINISHED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, taskId=task_1463517950564_0001_1_00_000000, startTime=1463517950834, finishTime=1463517950933, timeTaken=99, status=SUCCEEDED, successfulAttemptID=attempt_1463517950564_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950914, Process_Duration=4, Process_End_Time=1463517950918, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:50,934 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:50,934 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] : 1
2016-05-17 20:45:50,934 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,935 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:50,936 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:50,937 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_FINISHED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, vertexId=vertex_1463517950564_0001_1_00, initRequestedTime=1463517950767, initedTime=1463517950802, startRequestedTime=1463517950803, startedTime=1463517950803, finishTime=1463517950934, timeTaken=131, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950914, Process_Duration=4, Process_End_Time=1463517950918, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463517950834, firstTasksToStart=[ task_1463517950564_0001_1_00_000000 ], lastTaskFinishTime=1463517950932, lastTasksToFinish=[ task_1463517950564_0001_1_00_000000 ], minTaskDuration=98, maxTaskDuration=98, avgTaskDuration=98.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517950564_0001_1_00_000000 ], longestDurationTasks=[ task_1463517950564_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:50,938 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:50,938 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:50,944 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:50,944 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] attempt: attempt_1463517950564_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:50,944 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,944 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,945 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517950564_0001_00_000001, stoppedTime=1463517950945, exitStatus=0
2016-05-17 20:45:50,945 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:50,944 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517950914, Process_Duration=4, Process_End_Time=1463517950918, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 20:45:50,945 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:50,948 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:50,948 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,948 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:50,949 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517950564_0001_00_000001
2016-05-17 20:45:50,949 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_STARTED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, taskId=task_1463517950564_0001_1_01_000000, scheduledTime=1463517950949, launchTime=1463517950949
2016-05-17 20:45:50,949 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:50,949 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:50,950 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,951 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative, VertexName: FCF45131816E4FC5947C4DDFA51C1448, VertexParallelism: 1, TaskAttemptID:attempt_1463517950564_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B0968C4BAD2B45BF940D458AF443ECFD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=09D0E48849B241658D0742457F342690, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:50,951 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:50,951 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:50,952 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517950564_0001_00_000002
2016-05-17 20:45:50,952 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:50,953 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517950564_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:50,954 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:50,955 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517950564_0001_00_000002
2016-05-17 20:45:50,955 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:50,962 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517950564_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:50,962 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:50,963 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:50,963 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517950564_0001_00_000002, launchTime=1463517950962
2016-05-17 20:45:50,963 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517950564_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:50,964 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517950564_0001_00_000001, Exiting
2016-05-17 20:45:50,964 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517950564_0001_00_000001
2016-05-17 20:45:50,964 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517950564_0001_00_000001
2016-05-17 20:45:50,976 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517950564_0001_00_000002
2016-05-17 20:45:50,986 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517950564_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:50,987 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517950564_0001_1_01_000000_0] to container: [container_1463517950564_0001_00_000002]
2016-05-17 20:45:50,987 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:50,987 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517950564_0001_00_000002 given task: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,987 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517950564_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,988 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517950564_0001_1_01_000000_0] started. Is using containerId: [container_1463517950564_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:50,988 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0, startTime=1463517950988, containerId=container_1463517950564_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517950564_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:50,989 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:50,989 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:50 Starting to run new task attempt: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,989 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,989 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,990 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:50,990 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:50,990 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:50,990 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:50,990 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:50,990 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative, VertexName: FCF45131816E4FC5947C4DDFA51C1448, VertexParallelism: 1, TaskAttemptID:attempt_1463517950564_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B0968C4BAD2B45BF940D458AF443ECFD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=09D0E48849B241658D0742457F342690, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:50,991 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:50,991 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:50,987 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517950564_0001_00_000001 completed successfully
2016-05-17 20:45:50,992 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:50,993 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:50,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,004 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B0968C4BAD2B45BF940D458AF443ECFD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:51,005 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:51,005 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,006 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,006 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:51,007 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:51,007 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,009 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:51,016 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517950564_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:51,022 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=09D0E48849B241658D0742457F342690, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:51,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:51,026 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 09D0E48849B241658D0742457F342690
2016-05-17 20:45:51,027 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:51,027 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 09D0E48849B241658D0742457F342690
2016-05-17 20:45:51,045 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FCF45131816E4FC5947C4DDFA51C1448, ordinal: 1
2016-05-17 20:45:51,046 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCF45131816E4FC5947C4DDFA51C1448, mem on start (mb), free: 322, total: 346, max: 672
2016-05-17 20:45:51,047 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,047 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:51,048 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:51,055 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:51,055 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 09D0E48849B241658D0742457F342690
2016-05-17 20:45:51,055 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:51,055 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:51,055 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:51,055 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:51,056 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:51,056 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:51,056 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:51,056 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B0968C4BAD2B45BF940D458AF443ECFD, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:51,056 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 09D0E48849B241658D0742457F342690, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:51,056 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:51,056 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B0968C4BAD2B45BF940D458AF443ECFD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:51,057 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,057 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:51,057 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:51,058 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:51,058 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:51,060 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B0968C4BAD2B45BF940D458AF443ECFD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:51,061 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:51,061 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:51,061 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:51,062 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,065 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:51,066 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,068 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517950564_0001_1_00_000000_0_10034, runDuration: 64892, ]
2016-05-17 20:45:51,070 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517950564_0001_1_00_000000_0_10034] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463517951070, TimeTaken=1, Rate=0.19 MB/s
2016-05-17 20:45:51,070 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,070 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:51,077 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 266B1E76DC3542AB9322E27C80F9F0EE
2016-05-17 20:45:51,080 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:51,080 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"], id: 09D0E48849B241658D0742457F342690
2016-05-17 20:45:51,081 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B0968C4BAD2B45BF940D458AF443ECFD] #6 in 13ms
2016-05-17 20:45:51,082 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-17 20:45:51,083 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B0968C4BAD2B45BF940D458AF443ECFD] #6, status:false, isInterrupted:false
2016-05-17 20:45:51,083 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 266B1E76DC3542AB9322E27C80F9F0EE, for 1 inputs
2016-05-17 20:45:51,084 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:51,085 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 20:45:51,085 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:51,085 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:51,086 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 20:45:51,086 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,087 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:51,088 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FCF45131816E4FC5947C4DDFA51C1448, all 1 inputs ready in: 00:00:00.003
2016-05-17 20:45:51,090 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5f9bbec5; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@6a09a88c
2016-05-17 20:45:51,095 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:51,096 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:51,097 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,097 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517950564_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:51,099 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,099 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCF45131816E4FC5947C4DDFA51C1448, mem on close (mb), free: 319, total: 346, max: 672
2016-05-17 20:45:51,099 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,100 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:51,101 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:51,101 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:51,101 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:51,102 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:51 Completed running task attempt: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,106 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:51,107 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=363331584, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517951088, Process_Duration=10, Process_End_Time=1463517951098, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 20:45:51,108 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:51,108 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0, startTime=1463517950988, finishTime=1463517951107, timeTaken=119, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=363331584, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517951088, Process_Duration=10, Process_End_Time=1463517951098, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:51,109 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:51,110 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:51,110 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:51,110 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,111 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517950564_0001_00_000002
2016-05-17 20:45:51,111 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:51,111 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_FINISHED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, taskId=task_1463517950564_0001_1_01_000000, startTime=1463517950988, finishTime=1463517951110, timeTaken=122, status=SUCCEEDED, successfulAttemptID=attempt_1463517950564_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=363331584, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517951088, Process_Duration=10, Process_End_Time=1463517951098, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:51,111 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:51,111 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:51,112 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] : 1
2016-05-17 20:45:51,112 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:51,113 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:51,122 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_FINISHED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, vertexId=vertex_1463517950564_0001_1_01, initRequestedTime=1463517950769, initedTime=1463517950784, startRequestedTime=1463517950811, startedTime=1463517950811, finishTime=1463517951112, timeTaken=301, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=363331584, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517951088, Process_Duration=10, Process_End_Time=1463517951098, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463517950988, firstTasksToStart=[ task_1463517950564_0001_1_01_000000 ], lastTaskFinishTime=1463517951107, lastTasksToFinish=[ task_1463517950564_0001_1_01_000000 ], minTaskDuration=119, maxTaskDuration=119, avgTaskDuration=119.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517950564_0001_1_01_000000 ], longestDurationTasks=[ task_1463517950564_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:51,129 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:51,133 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:51,133 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517950564_0001_00_000002, stoppedTime=1463517951133, exitStatus=0
2016-05-17 20:45:51,133 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:51,134 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:51,134 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:51,135 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517950564_0001_1
2016-05-17 20:45:51,136 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517950564_0001_1
2016-05-17 20:45:51,136 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 09D0E48849B241658D0742457F342690 for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:51,141 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517950564_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:51,141 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517950564_0001_00_000002: 30 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:51,142 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517950564_0001_00_000002, Exiting
2016-05-17 20:45:51,142 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517950564_0001_00_000002
2016-05-17 20:45:51,142 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517950564_0001_00_000002
2016-05-17 20:45:51,146 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517950564_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:51,146 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517950564_0001_00_000002 completed successfully
2016-05-17 20:45:51,165 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:51,169 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517950564_0001_1, queueSize=0
2016-05-17 20:45:51,169 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:51,170 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517950564_0001_1, startTime=1463517950766, finishTime=1463517951165, timeTaken=399, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=896, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=713031680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035902002, Process_Duration=14, Process_End_Time=2927035902016, Read_Duration=3, Tuples_Read=20, Tuples_Written=18, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 20:45:51,170 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517950564_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:51,171 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517950564_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:51,171 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517950564_0001_00_000002
2016-05-17 20:45:51,171 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:51 Completed Dag: dag_1463517950564_0001_1
2016-05-17 20:45:51,172 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517950564_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:51,173 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517950564_0001_1_post
2016-05-17 20:45:51,173 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:51,173 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:51,189 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463517950564_0001
2016-05-17 20:45:51,192 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:51,192 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:51,192 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:51,192 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:51,192 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:51,193 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:51,193 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:51,195 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:51,195 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:51,196 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:51,196 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:51,196 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:51,196 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:51,204 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001
2016-05-17 20:45:51,205 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative/_temporary
2016-05-17 20:45:51,205 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.662
2016-05-17 20:45:51,320 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-17 20:45:51,450 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:51,451 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:51,503 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 20:45:51,519 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-17 20:45:51,519 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:51,519 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:51,520 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:51,537 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 20:45:51,553 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-17 20:45:51,603 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:51,603 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:51,605 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-17 20:45:51,605 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:51,605 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:51,605 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:51,608 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51,608 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:51,621 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:51,625 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:51,626 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:51,626 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:51,637 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001 doesn't exist and is created
2016-05-17 20:45:51,654 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:51,654 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:51,655 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001
2016-05-17 20:45:51,658 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517951626_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:51,669 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:51,670 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:51,671 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:51,672 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:51,672 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001/localmode-log-dir
2016-05-17 20:45:51,672 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001/localmode-log-dir/history.txt.appattempt_1463517951626_0001_000000, maxErrors=10
2016-05-17 20:45:51,673 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:51,673 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517951626_0001_000000, appSubmitTime=1463517951658, launchTime=1463517951658
2016-05-17 20:45:51,677 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:51,679 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:51,680 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:51,680 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:51,682 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48409
2016-05-17 20:45:51,684 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:51,688 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517951626_0001_000000, startTime=1463517951688
2016-05-17 20:45:51,688 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:51,688 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:51,754 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:51,754 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:51,755 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463517951626_0001, dagName=[7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51,775 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51,777 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517951626_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001/localmode-log-dir/dag_1463517951626_0001_1.dot
2016-05-17 20:45:51,777 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001/localmode-log-dir/dag_1463517951626_0001_1-tez-dag.pb.txt
2016-05-17 20:45:51,787 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517951626_0001_1
2016-05-17 20:45:51,787 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:51,787 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51 Running Dag: dag_1463517951626_0001_1
2016-05-17 20:45:51,801 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:51,801 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517951626_0001_1, submitTime=1463517951775
2016-05-17 20:45:51,803 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,803 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:51,803 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:51,804 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517951626_0001_1, initTime=1463517951801
2016-05-17 20:45:51,805 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517951626_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:51,805 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_STARTED]: dagID=dag_1463517951626_0001_1, startTime=1463517951805
2016-05-17 20:45:51,806 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:51,805 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:51,806 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517951626_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:51,806 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B29AD20FE1E94CDB96DBAD67628E7E85 : {27D522ED170E40519DBAB82CD01BAE88={InputName=27D522ED170E40519DBAB82CD01BAE88}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:51,806 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 27D522ED170E40519DBAB82CD01BAE88, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:51,806 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,806 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,806 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,807 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]. Starting root input initializers: 1
2016-05-17 20:45:51,807 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463517951626_0001, dagName=[7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51,807 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:51,807 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,808 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 27D522ED170E40519DBAB82CD01BAE88 on vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,810 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:51,810 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:51,810 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,811 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,811 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,811 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8C53548AE7C04A28A8B070108FE52298, vertexId=vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:51,812 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8C53548AE7C04A28A8B070108FE52298, vertexId=vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,810 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 27D522ED170E40519DBAB82CD01BAE88 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:51,814 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:51,814 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8C53548AE7C04A28A8B070108FE52298, vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:51,814 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8C53548AE7C04A28A8B070108FE52298, vertexId=vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,815 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:51,816 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:51,817 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, vertexId=vertex_1463517951626_0001_1_01, initRequestedTime=1463517951807, initedTime=1463517951816, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:51,818 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:51,823 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517951626_0001, with dag id: dag_1463517951626_0001_1
2016-05-17 20:45:51,824 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:51,831 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 27D522ED170E40519DBAB82CD01BAE88 on vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,832 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:51,833 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {27D522ED170E40519DBAB82CD01BAE88=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:51,834 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] parallelism set to 1
2016-05-17 20:45:51,834 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:51,835 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517951626_0001_1_00, eventCount=1
2016-05-17 20:45:51,836 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:51,838 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, vertexId=vertex_1463517951626_0001_1_00, initRequestedTime=1463517951806, initedTime=1463517951836, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:51,838 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] with distanceFromRoot: 0
2016-05-17 20:45:51,839 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:51,839 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:51,839 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,840 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:51,840 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517951626_0001_1_00, startRequestedTime=1463517951839, startedTime=1463517951839
2016-05-17 20:45:51,840 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:51,841 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:51,841 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_STARTED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, taskId=task_1463517951626_0001_1_00_000000, scheduledTime=1463517951840, launchTime=1463517951840
2016-05-17 20:45:51,841 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:51,841 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517951626_0001_1_00 for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] numStartedSources: 1 numSources: 1
2016-05-17 20:45:51,841 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] with distanceFromRoot: 1
2016-05-17 20:45:51,842 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,842 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B29AD20FE1E94CDB96DBAD67628E7E85 in vertex: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,847 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F32E0FE10D22481EBADD54A3C5B6E7B5 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:51,848 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:51,848 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517951626_0001_1_01, startRequestedTime=1463517951842, startedTime=1463517951842
2016-05-17 20:45:51,848 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:51,848 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult, VertexName: B29AD20FE1E94CDB96DBAD67628E7E85, VertexParallelism: 1, TaskAttemptID:attempt_1463517951626_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=27D522ED170E40519DBAB82CD01BAE88, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:51,849 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:51,849 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:51,850 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:51,851 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:51,851 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:51,852 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517951626_0001_00_000001
2016-05-17 20:45:51,852 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:51,854 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517951626_0001_00_000001
2016-05-17 20:45:51,854 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:51,855 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:51,855 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:51,856 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517951626_0001_00_000001, launchTime=1463517951855
2016-05-17 20:45:51,856 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517951626_0001_00_000001
2016-05-17 20:45:51,857 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517951626_0001_1_00_000000_0] to container: [container_1463517951626_0001_00_000001]
2016-05-17 20:45:51,858 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:51,858 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517951626_0001_1_00_000000_0] started. Is using containerId: [container_1463517951626_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:51,859 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0, startTime=1463517951858, containerId=container_1463517951626_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517951626_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:51,859 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:51,859 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:51,859 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:51,860 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517951626_0001_00_000001 given task: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,860 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517951626_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51 Starting to run new task attempt: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,861 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,861 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:51,861 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:51,861 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:51,861 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:51,861 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:51,861 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:51,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult, VertexName: B29AD20FE1E94CDB96DBAD67628E7E85, VertexParallelism: 1, TaskAttemptID:attempt_1463517951626_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=27D522ED170E40519DBAB82CD01BAE88, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:51,862 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:51,864 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,864 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:51,865 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,866 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=27D522ED170E40519DBAB82CD01BAE88, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:51,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:51,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,868 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,874 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,874 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:51,875 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:51,875 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,876 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:51,876 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,877 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,877 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:51,878 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:51,878 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,881 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517951626_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:51,881 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,890 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B29AD20FE1E94CDB96DBAD67628E7E85, ordinal: 0
2016-05-17 20:45:51,890 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B29AD20FE1E94CDB96DBAD67628E7E85, mem on start (mb), free: 225, total: 346, max: 672
2016-05-17 20:45:51,890 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,891 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:51,891 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:51,891 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:51,892 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:51,892 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:51,892 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:51,893 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:51,893 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:51,894 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:51,894 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 27D522ED170E40519DBAB82CD01BAE88, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:51,894 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F32E0FE10D22481EBADD54A3C5B6E7B5, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:51,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:51,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,895 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:51,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,896 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 27D522ED170E40519DBAB82CD01BAE88 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:51,896 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:51,897 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:51,897 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:51,898 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,901 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:51,902 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:51,903 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@42e0615c
2016-05-17 20:45:51,904 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,904 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: A64C9A1178744E92B0B90EA7911B1CFF
2016-05-17 20:45:51,904 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] A64C9A1178744E92B0B90EA7911B1CFF
2016-05-17 20:45:51,905 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@42e0615c
2016-05-17 20:45:51,912 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:51,912 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:51,915 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@11c76237; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@390111ff; comparator=cascading.tuple.hadoop.util.TupleComparator@75449995; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:51,916 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:51,916 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:51,917 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:51,930 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:51,931 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:51,931 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:51,931 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:51,932 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:52,015 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:52,016 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:52,301 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:52,301 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:52,302 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B29AD20FE1E94CDB96DBAD67628E7E85, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:52,306 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:52,306 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B29AD20FE1E94CDB96DBAD67628E7E85, mem on close (mb), free: 416, total: 541, max: 672
2016-05-17 20:45:52,307 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:52,307 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:52,307 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 22; bufvoid = 104857600
2016-05-17 20:45:52,308 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2016-05-17 20:45:52,311 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:52 Completed running task attempt: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:52,317 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:52,318 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517951626_0001_1_00, eventCount=1
2016-05-17 20:45:52,318 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:52,318 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0, startTime=1463517951858, finishTime=1463517952318, timeTaken=460, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952302, Process_Duration=4, Process_End_Time=1463517952306, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:52,319 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:52,319 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:52,319 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:52,320 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_FINISHED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, taskId=task_1463517951626_0001_1_00_000000, startTime=1463517951858, finishTime=1463517952319, timeTaken=461, status=SUCCEEDED, successfulAttemptID=attempt_1463517951626_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952302, Process_Duration=4, Process_End_Time=1463517952306, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:52,320 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:52,320 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] : 1
2016-05-17 20:45:52,320 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:52,321 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:52,323 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:52,322 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_FINISHED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, vertexId=vertex_1463517951626_0001_1_00, initRequestedTime=1463517951806, initedTime=1463517951836, startRequestedTime=1463517951839, startedTime=1463517951839, finishTime=1463517952321, timeTaken=482, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952302, Process_Duration=4, Process_End_Time=1463517952306, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517951858, firstTasksToStart=[ task_1463517951626_0001_1_00_000000 ], lastTaskFinishTime=1463517952318, lastTasksToFinish=[ task_1463517951626_0001_1_00_000000 ], minTaskDuration=460, maxTaskDuration=460, avgTaskDuration=460.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517951626_0001_1_00_000000 ], longestDurationTasks=[ task_1463517951626_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:52,324 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:52,325 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:52,325 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] attempt: attempt_1463517951626_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:52,326 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:52,326 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:52,326 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517951626_0001_00_000001, stoppedTime=1463517952326, exitStatus=0
2016-05-17 20:45:52,327 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:52,327 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:52,326 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:52,328 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:52,329 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_STARTED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, taskId=task_1463517951626_0001_1_01_000000, scheduledTime=1463517952328, launchTime=1463517952328
2016-05-17 20:45:52,329 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:52,329 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:52,330 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517951626_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:52,330 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult, VertexName: F32E0FE10D22481EBADD54A3C5B6E7B5, VertexParallelism: 1, TaskAttemptID:attempt_1463517951626_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B29AD20FE1E94CDB96DBAD67628E7E85, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8C53548AE7C04A28A8B070108FE52298, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:52,331 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:52,331 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:52,331 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=2, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517952302, Process_Duration=4, Process_End_Time=1463517952306, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:52,332 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:52,332 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:52,332 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,335 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517951626_0001_00_000001
2016-05-17 20:45:52,336 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517951626_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:52,336 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517951626_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:52,335 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517951626_0001_00_000002
2016-05-17 20:45:52,337 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:52,338 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517951626_0001_00_000001, Exiting
2016-05-17 20:45:52,338 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517951626_0001_00_000001
2016-05-17 20:45:52,338 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517951626_0001_00_000001
2016-05-17 20:45:52,339 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517951626_0001_00_000002
2016-05-17 20:45:52,341 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:52,342 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517951626_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:52,343 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517951626_0001_00_000001 completed successfully
2016-05-17 20:45:52,344 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:52,345 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:52,346 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:52,348 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517951626_0001_00_000002, launchTime=1463517952346
2016-05-17 20:45:52,349 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517951626_0001_00_000002
2016-05-17 20:45:52,350 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517951626_0001_1_01_000000_0] to container: [container_1463517951626_0001_00_000002]
2016-05-17 20:45:52,350 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:52,351 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517951626_0001_1_01_000000_0] started. Is using containerId: [container_1463517951626_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:52,352 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0, startTime=1463517952351, containerId=container_1463517951626_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517951626_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:52,352 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:52,352 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:52,352 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:52,354 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517951626_0001_00_000002 given task: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,355 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517951626_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52 Starting to run new task attempt: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,355 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,356 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,356 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:52,356 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:52,356 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:52,356 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:52,356 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:52,356 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult, VertexName: F32E0FE10D22481EBADD54A3C5B6E7B5, VertexParallelism: 1, TaskAttemptID:attempt_1463517951626_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B29AD20FE1E94CDB96DBAD67628E7E85, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8C53548AE7C04A28A8B070108FE52298, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:52,357 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:52,361 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,362 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:52,363 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,364 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517951626_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:52,365 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B29AD20FE1E94CDB96DBAD67628E7E85, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:52,366 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:52,367 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,367 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8C53548AE7C04A28A8B070108FE52298, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:52,367 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,369 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:52,370 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8C53548AE7C04A28A8B070108FE52298
2016-05-17 20:45:52,370 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:52,370 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8C53548AE7C04A28A8B070108FE52298
2016-05-17 20:45:52,371 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:52,371 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:52,372 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:52,374 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,375 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,388 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:52,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8C53548AE7C04A28A8B070108FE52298
2016-05-17 20:45:52,388 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F32E0FE10D22481EBADD54A3C5B6E7B5, ordinal: 1
2016-05-17 20:45:52,388 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F32E0FE10D22481EBADD54A3C5B6E7B5, mem on start (mb), free: 409, total: 541, max: 672
2016-05-17 20:45:52,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:52,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:52,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:52,389 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:52,389 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:52,389 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:52,389 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:52,389 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:52,389 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:52,390 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8C53548AE7C04A28A8B070108FE52298, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:52,390 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B29AD20FE1E94CDB96DBAD67628E7E85, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:52,390 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:52,390 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,390 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B29AD20FE1E94CDB96DBAD67628E7E85 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:52,391 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:52,393 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:52,393 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:52,393 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:52,393 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B29AD20FE1E94CDB96DBAD67628E7E85 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:52,393 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:52,394 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:52,394 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:52,395 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,395 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:52,396 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,398 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517951626_0001_1_00_000000_0_10040, runDuration: 439431, ]
2016-05-17 20:45:52,399 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:52,400 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: A64C9A1178744E92B0B90EA7911B1CFF
2016-05-17 20:45:52,400 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"], id: 8C53548AE7C04A28A8B070108FE52298
2016-05-17 20:45:52,401 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-17 20:45:52,401 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] A64C9A1178744E92B0B90EA7911B1CFF, for 1 inputs
2016-05-17 20:45:52,401 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517951626_0001_1_00_000000_0_10040] to DISK_DIRECT, CompressedSize=36, DecompressedSize=32,EndTime=1463517952401, TimeTaken=2, Rate=0.02 MB/s
2016-05-17 20:45:52,402 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,402 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:52,402 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B29AD20FE1E94CDB96DBAD67628E7E85] #7 in 3ms
2016-05-17 20:45:52,402 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B29AD20FE1E94CDB96DBAD67628E7E85] #7, status:false, isInterrupted:false
2016-05-17 20:45:52,403 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:52,404 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 36 bytes from disk
2016-05-17 20:45:52,404 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:52,404 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:52,404 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2016-05-17 20:45:52,404 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,405 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F32E0FE10D22481EBADD54A3C5B6E7B5, all 1 inputs ready in: 00:00:00.003
2016-05-17 20:45:52,405 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:52,407 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4ce1ffdd; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@9652d9a
2016-05-17 20:45:52,408 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:52,411 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:52,412 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,412 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517951626_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:52,414 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,414 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F32E0FE10D22481EBADD54A3C5B6E7B5, mem on close (mb), free: 407, total: 541, max: 672
2016-05-17 20:45:52,414 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,414 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:52,415 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:52,415 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:52,416 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:52,416 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:52 Completed running task attempt: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,417 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:52,419 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0, startTime=1463517952351, finishTime=1463517952418, timeTaken=67, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952405, Process_Duration=9, Process_End_Time=1463517952414, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-05-17 20:45:52,419 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:52,419 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:52,419 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,419 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:52,420 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_FINISHED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, taskId=task_1463517951626_0001_1_01_000000, startTime=1463517952351, finishTime=1463517952419, timeTaken=68, status=SUCCEEDED, successfulAttemptID=attempt_1463517951626_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952405, Process_Duration=9, Process_End_Time=1463517952414, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-05-17 20:45:52,420 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:52,420 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] : 1
2016-05-17 20:45:52,421 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:52,420 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:52,422 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:52,422 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_FINISHED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, vertexId=vertex_1463517951626_0001_1_01, initRequestedTime=1463517951807, initedTime=1463517951816, startRequestedTime=1463517951842, startedTime=1463517951842, finishTime=1463517952421, timeTaken=579, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952405, Process_Duration=9, Process_End_Time=1463517952414, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2, vertexStats=firstTaskStartTime=1463517952351, firstTasksToStart=[ task_1463517951626_0001_1_01_000000 ], lastTaskFinishTime=1463517952418, lastTasksToFinish=[ task_1463517951626_0001_1_01_000000 ], minTaskDuration=67, maxTaskDuration=67, avgTaskDuration=67.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517951626_0001_1_01_000000 ], longestDurationTasks=[ task_1463517951626_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:52,422 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:52,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517952405, Process_Duration=9, Process_End_Time=1463517952414, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=2]]
2016-05-17 20:45:52,423 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:52,423 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,423 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:52,424 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517951626_0001_00_000002, stoppedTime=1463517952424, exitStatus=0
2016-05-17 20:45:52,425 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:52,425 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:52,426 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517951626_0001_1
2016-05-17 20:45:52,424 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517951626_0001_00_000002
2016-05-17 20:45:52,427 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517951626_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:52,427 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517951626_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:52,427 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517951626_0001_00_000002, Exiting
2016-05-17 20:45:52,427 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517951626_0001_00_000002
2016-05-17 20:45:52,428 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517951626_0001_00_000002
2016-05-17 20:45:52,427 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517951626_0001_1
2016-05-17 20:45:52,424 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:52,429 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8C53548AE7C04A28A8B070108FE52298 for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:52,429 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517951626_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:52,429 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517951626_0001_00_000002 completed successfully
2016-05-17 20:45:52,439 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:52,442 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517951626_0001_1, queueSize=0
2016-05-17 20:45:52,442 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:52,443 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517951626_0001_1, startTime=1463517951805, finishTime=1463517952439, timeTaken=634, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=130, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1134559232, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035904707, Process_Duration=13, Process_End_Time=2927035904720, Read_Duration=2, Tuples_Read=7, Tuples_Written=4, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=2
2016-05-17 20:45:52,443 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517951626_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:52,443 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517951626_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:52,444 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517951626_0001_00_000002
2016-05-17 20:45:52,444 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:52 Completed Dag: dag_1463517951626_0001_1
2016-05-17 20:45:52,446 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517951626_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:52,446 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517951626_0001_1_post
2016-05-17 20:45:52,447 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,447 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:52,452 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463517951626_0001
2016-05-17 20:45:52,455 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:52,455 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:52,456 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:52,456 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:52,457 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:52,458 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:52,458 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:52,460 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:52,461 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:52,461 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:52,461 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:52,464 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:52,464 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:52,475 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001
2016-05-17 20:45:52,476 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult/_temporary
2016-05-17 20:45:52,477 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.872
2016-05-17 20:45:52,515 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-17 20:45:52,541 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:52,541 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:52,577 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 20:45:52,585 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-17 20:45:52,585 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:52,586 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:52,586 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:52,607 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:52,608 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-17 20:45:52,621 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:52,621 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:52,621 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-17 20:45:52,622 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:52,622 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:52,622 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:52,624 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52,625 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:52,637 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:52,641 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:52,642 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:52,642 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:52,651 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001 doesn't exist and is created
2016-05-17 20:45:52,668 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:52,668 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:52,669 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001
2016-05-17 20:45:52,673 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517952642_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:52,683 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:52,685 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:52,687 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:52,688 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:52,689 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001/localmode-log-dir
2016-05-17 20:45:52,689 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001/localmode-log-dir/history.txt.appattempt_1463517952642_0001_000000, maxErrors=10
2016-05-17 20:45:52,689 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:52,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517952642_0001_000000, appSubmitTime=1463517952672, launchTime=1463517952672
2016-05-17 20:45:52,698 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:52,698 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46736
2016-05-17 20:45:52,700 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:52,702 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:52,702 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:52,703 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:52,705 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517952642_0001_000000, startTime=1463517952705
2016-05-17 20:45:52,706 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:52,706 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:52,768 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:52,768 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:52,768 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463517952642_0001, dagName=[698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52,784 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52,786 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517952642_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001/localmode-log-dir/dag_1463517952642_0001_1.dot
2016-05-17 20:45:52,787 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001/localmode-log-dir/dag_1463517952642_0001_1-tez-dag.pb.txt
2016-05-17 20:45:52,795 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517952642_0001_1
2016-05-17 20:45:52,795 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,795 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52 Running Dag: dag_1463517952642_0001_1
2016-05-17 20:45:52,807 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517952642_0001_1, submitTime=1463517952785
2016-05-17 20:45:52,807 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:52,808 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FC38A02B300045A5AACF847FDC408ADB
2016-05-17 20:45:52,808 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:52,808 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517952642_0001_1, initTime=1463517952807
2016-05-17 20:45:52,808 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:52,809 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517952642_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:52,809 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_STARTED]: dagID=dag_1463517952642_0001_1, startTime=1463517952809
2016-05-17 20:45:52,810 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:52,810 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517952642_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:52,810 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:52,810 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FC38A02B300045A5AACF847FDC408ADB : {30973D9C33E44B84B0BC1B2C410B42D3={InputName=30973D9C33E44B84B0BC1B2C410B42D3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:52,810 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 30973D9C33E44B84B0BC1B2C410B42D3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:52,810 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,811 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,811 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,812 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]. Starting root input initializers: 1
2016-05-17 20:45:52,812 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:52,813 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 30973D9C33E44B84B0BC1B2C410B42D3 on vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,814 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:52,815 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 30973D9C33E44B84B0BC1B2C410B42D3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:52,815 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:52,815 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:52,815 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463517952642_0001, dagName=[698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52,817 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:52,818 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 30973D9C33E44B84B0BC1B2C410B42D3 on vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,818 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:52,818 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {30973D9C33E44B84B0BC1B2C410B42D3=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:52,819 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] parallelism set to 1
2016-05-17 20:45:52,819 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:52,820 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517952642_0001_1_00, eventCount=1
2016-05-17 20:45:52,820 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,820 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6515355947F84C98A03A787DA01532ED, vertexId=vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:52,820 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6515355947F84C98A03A787DA01532ED, vertexId=vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,822 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6515355947F84C98A03A787DA01532ED, vertexName=FC38A02B300045A5AACF847FDC408ADB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:52,822 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6515355947F84C98A03A787DA01532ED, vertexId=vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,823 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:52,823 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, vertexId=vertex_1463517952642_0001_1_00, initRequestedTime=1463517952810, initedTime=1463517952823, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:52,824 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] with distanceFromRoot: 0
2016-05-17 20:45:52,824 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:52,824 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FC38A02B300045A5AACF847FDC408ADB
2016-05-17 20:45:52,824 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,825 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:52,825 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517952642_0001_1_00, startRequestedTime=1463517952824, startedTime=1463517952824
2016-05-17 20:45:52,825 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:52,826 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:52,826 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:TASK_STARTED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, taskId=task_1463517952642_0001_1_00_000000, scheduledTime=1463517952825, launchTime=1463517952825
2016-05-17 20:45:52,826 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517952642_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:52,826 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,827 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall, VertexName: FC38A02B300045A5AACF847FDC408ADB, VertexParallelism: 1, TaskAttemptID:attempt_1463517952642_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=30973D9C33E44B84B0BC1B2C410B42D3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6515355947F84C98A03A787DA01532ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:52,827 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517952642_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:52,827 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:52,828 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:52,829 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:52,829 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:52,829 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517952642_0001_00_000001
2016-05-17 20:45:52,829 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:52,831 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517952642_0001_00_000001
2016-05-17 20:45:52,831 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:52,831 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517952642_0001, with dag id: dag_1463517952642_0001_1
2016-05-17 20:45:52,832 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:52,833 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:52,833 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517952642_0001_00_000001, launchTime=1463517952833
2016-05-17 20:45:52,834 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517952642_0001_00_000001
2016-05-17 20:45:52,835 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517952642_0001_1_00_000000_0] to container: [container_1463517952642_0001_00_000001]
2016-05-17 20:45:52,835 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:52,836 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517952642_0001_00_000001 given task: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,836 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517952642_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,836 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517952642_0001_1_00_000000_0] started. Is using containerId: [container_1463517952642_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:52,837 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:52 Starting to run new task attempt: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,837 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0, startTime=1463517952836, containerId=container_1463517952642_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517952642_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:52,837 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,837 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,838 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:52,838 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:52,837 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517952642_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:52,838 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517952642_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:52,838 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:52,839 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:52,839 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:52,839 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall, VertexName: FC38A02B300045A5AACF847FDC408ADB, VertexParallelism: 1, TaskAttemptID:attempt_1463517952642_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=30973D9C33E44B84B0BC1B2C410B42D3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6515355947F84C98A03A787DA01532ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:52,839 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:52,840 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,840 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:52,840 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,840 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517952642_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:52,841 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=30973D9C33E44B84B0BC1B2C410B42D3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:52,842 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6515355947F84C98A03A787DA01532ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:52,841 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,842 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:52,842 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,843 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,844 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:52,845 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:52,845 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,842 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:52,847 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6515355947F84C98A03A787DA01532ED
2016-05-17 20:45:52,847 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:52,848 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6515355947F84C98A03A787DA01532ED
2016-05-17 20:45:52,855 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FC38A02B300045A5AACF847FDC408ADB, ordinal: 0
2016-05-17 20:45:52,855 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FC38A02B300045A5AACF847FDC408ADB, mem on start (mb), free: 342, total: 541, max: 672
2016-05-17 20:45:52,855 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,856 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:52,856 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:52,860 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:52,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6515355947F84C98A03A787DA01532ED
2016-05-17 20:45:52,860 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:52,861 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:52,861 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:52,861 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:52,861 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:52,861 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:52,861 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:52,861 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:52,861 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 30973D9C33E44B84B0BC1B2C410B42D3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:52,862 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6515355947F84C98A03A787DA01532ED, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:52,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:52,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 30973D9C33E44B84B0BC1B2C410B42D3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:52,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,862 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:52,863 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,862 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:52,863 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:52,863 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:52,863 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,864 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:52,865 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@186f5f95
2016-05-17 20:45:52,866 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@186f5f95
2016-05-17 20:45:52,866 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:52,866 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:52,866 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:52,867 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,867 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"], id: 6515355947F84C98A03A787DA01532ED
2016-05-17 20:45:52,867 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-17 20:45:52,868 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:52,868 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:52,868 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FC38A02B300045A5AACF847FDC408ADB, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:52,870 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:52,870 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:52,870 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,870 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517952642_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:52,871 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,871 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FC38A02B300045A5AACF847FDC408ADB, mem on close (mb), free: 340, total: 541, max: 672
2016-05-17 20:45:52 Completed running task attempt: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,872 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:52,873 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 20:45:52,873 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:52,873 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0, startTime=1463517952836, finishTime=1463517952873, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:52,873 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:52,874 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517952642_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:52,874 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,874 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:52,874 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,874 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517952642_0001_00_000001
2016-05-17 20:45:52,875 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:TASK_FINISHED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, taskId=task_1463517952642_0001_1_00_000000, startTime=1463517952836, finishTime=1463517952874, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463517952642_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:52,875 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517952642_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:52,875 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:52,876 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] : 1
2016-05-17 20:45:52,876 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,877 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:52,877 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:VERTEX_FINISHED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, vertexId=vertex_1463517952642_0001_1_00, initRequestedTime=1463517952810, initedTime=1463517952823, startRequestedTime=1463517952824, startedTime=1463517952824, finishTime=1463517952876, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463517952836, firstTasksToStart=[ task_1463517952642_0001_1_00_000000 ], lastTaskFinishTime=1463517952873, lastTasksToFinish=[ task_1463517952642_0001_1_00_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517952642_0001_1_00_000000 ], longestDurationTasks=[ task_1463517952642_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:52,877 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:52,877 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:52,878 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517952642_0001_00_000001, stoppedTime=1463517952878, exitStatus=0
2016-05-17 20:45:52,878 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:52,878 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:52,878 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:52,878 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517952642_0001_1
2016-05-17 20:45:52,878 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517952642_0001_1
2016-05-17 20:45:52,878 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6515355947F84C98A03A787DA01532ED for vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,885 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517952642_0001_1, queueSize=0
2016-05-17 20:45:52,886 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:52,886 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517952642_0001_1, startTime=1463517952809, finishTime=1463517952885, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:52,886 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517952642_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:52,886 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517952642_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:52,886 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517952642_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:52 Completed Dag: dag_1463517952642_0001_1
2016-05-17 20:45:52,887 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517952642_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:52,887 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517952642_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:52,887 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517952642_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:52,887 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517952642_0001_1_post
2016-05-17 20:45:52,887 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,887 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:52,887 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517952642_0001_00_000001, Exiting
2016-05-17 20:45:52,888 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517952642_0001_00_000001
2016-05-17 20:45:52,888 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517952642_0001_00_000001
2016-05-17 20:45:52,888 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:52,890 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517952642_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:52,890 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517952642_0001_00_000001 completed successfully
2016-05-17 20:45:52,891 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:52,895 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463517952642_0001
2016-05-17 20:45:52,898 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:52,899 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:52,899 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:52,899 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:52,899 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:52,899 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:52,900 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:52,900 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:52,900 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:52,901 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:52,901 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:52,901 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:52,902 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:52,905 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001
2016-05-17 20:45:52,906 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall/_temporary
2016-05-17 20:45:52,906 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.285
2016-05-17 20:45:52,939 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-17 20:45:52,963 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:52,963 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:52,986 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:52,986 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:53,002 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 20:45:53,018 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 20:45:53,018 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:53,018 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:53,018 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:53,034 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 20:45:53,040 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-17 20:45:53,086 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:53,086 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:53,086 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-17 20:45:53,086 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:53,087 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:53,087 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:53,088 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53,088 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:53,099 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:53,104 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:53,104 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:53,104 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:53,113 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001 doesn't exist and is created
2016-05-17 20:45:53,129 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:53,130 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:53,130 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001
2016-05-17 20:45:53,133 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517953104_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:53,143 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:53,144 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:53,145 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:53,146 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:53,146 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001/localmode-log-dir
2016-05-17 20:45:53,146 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001/localmode-log-dir/history.txt.appattempt_1463517953104_0001_000000, maxErrors=10
2016-05-17 20:45:53,146 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:53,147 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517953104_0001_000000, appSubmitTime=1463517953133, launchTime=1463517953133
2016-05-17 20:45:53,155 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:53,156 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:53,157 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:53,157 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:53,163 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49472
2016-05-17 20:45:53,165 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:53,166 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517953104_0001_000000, startTime=1463517953166
2016-05-17 20:45:53,166 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:53,166 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:53,230 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:53,230 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:53,230 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463517953104_0001, dagName=[FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53,247 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53,248 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517953104_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001/localmode-log-dir/dag_1463517953104_0001_1.dot
2016-05-17 20:45:53,249 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001/localmode-log-dir/dag_1463517953104_0001_1-tez-dag.pb.txt
2016-05-17 20:45:53,264 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517953104_0001_1
2016-05-17 20:45:53,265 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,265 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53 Running Dag: dag_1463517953104_0001_1
2016-05-17 20:45:53,278 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:53,278 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517953104_0001_1, submitTime=1463517953247
2016-05-17 20:45:53,279 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,279 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:53,279 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:53,280 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517953104_0001_1, initTime=1463517953279
2016-05-17 20:45:53,280 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953104_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:53,280 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_STARTED]: dagID=dag_1463517953104_0001_1, startTime=1463517953280
2016-05-17 20:45:53,281 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:53,281 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953104_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:53,281 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BEB67DFC5DAB4C0D930D58116C1C92B7 : {C9009DA06C044CBCABD8DC9B87438BBD={InputName=C9009DA06C044CBCABD8DC9B87438BBD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:53,281 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C9009DA06C044CBCABD8DC9B87438BBD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:53,281 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,281 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,282 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,282 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]. Starting root input initializers: 1
2016-05-17 20:45:53,280 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:53,281 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463517953104_0001, dagName=[FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53,282 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:53,282 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,283 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C9009DA06C044CBCABD8DC9B87438BBD on vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,284 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:53,285 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C9009DA06C044CBCABD8DC9B87438BBD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:53,285 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:53,285 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:53,285 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:53,286 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,286 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,286 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,286 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=24E8A8ECAD434F3F811665337F82343F, vertexId=vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:53,286 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=24E8A8ECAD434F3F811665337F82343F, vertexId=vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,287 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:53,288 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C9009DA06C044CBCABD8DC9B87438BBD on vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,288 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=24E8A8ECAD434F3F811665337F82343F, vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:53,288 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=24E8A8ECAD434F3F811665337F82343F, vertexId=vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,289 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,289 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, vertexId=vertex_1463517953104_0001_1_01, initRequestedTime=1463517953282, initedTime=1463517953289, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:53,289 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:53,290 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,290 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C9009DA06C044CBCABD8DC9B87438BBD=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:53,290 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] parallelism set to 1
2016-05-17 20:45:53,290 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,290 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953104_0001_1_00, eventCount=1
2016-05-17 20:45:53,291 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, vertexId=vertex_1463517953104_0001_1_00, initRequestedTime=1463517953281, initedTime=1463517953291, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:53,291 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] with distanceFromRoot: 0
2016-05-17 20:45:53,291 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:53,291 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,291 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,291 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,292 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,292 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953104_0001_1_00, startRequestedTime=1463517953291, startedTime=1463517953291
2016-05-17 20:45:53,292 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,293 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:53,293 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_STARTED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, taskId=task_1463517953104_0001_1_00_000000, scheduledTime=1463517953292, launchTime=1463517953292
2016-05-17 20:45:53,293 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:53,293 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517953104_0001_1_00 for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] numStartedSources: 1 numSources: 1
2016-05-17 20:45:53,293 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] with distanceFromRoot: 1
2016-05-17 20:45:53,293 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,293 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BEB67DFC5DAB4C0D930D58116C1C92B7 in vertex: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,294 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F4FD1F2D4E2B402BA1A24AC7D23055F6 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:53,294 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,294 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953104_0001_1_01, startRequestedTime=1463517953293, startedTime=1463517953293
2016-05-17 20:45:53,294 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,294 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown, VertexName: BEB67DFC5DAB4C0D930D58116C1C92B7, VertexParallelism: 1, TaskAttemptID:attempt_1463517953104_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C9009DA06C044CBCABD8DC9B87438BBD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:53,294 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:53,295 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:53,296 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:53,296 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:53,296 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:53,297 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953104_0001_00_000001
2016-05-17 20:45:53,297 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:53,298 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517953104_0001_00_000001
2016-05-17 20:45:53,299 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:53,299 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517953104_0001, with dag id: dag_1463517953104_0001_1
2016-05-17 20:45:53,300 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:53,300 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:53,300 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953104_0001_00_000001, launchTime=1463517953300
2016-05-17 20:45:53,301 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953104_0001_00_000001
2016-05-17 20:45:53,302 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953104_0001_1_00_000000_0] to container: [container_1463517953104_0001_00_000001]
2016-05-17 20:45:53,302 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:53,302 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953104_0001_00_000001 given task: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,302 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953104_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,303 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953104_0001_1_00_000000_0] started. Is using containerId: [container_1463517953104_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:53,303 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0, startTime=1463517953302, containerId=container_1463517953104_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953104_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:53,303 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:53,303 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:53,303 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:53 Starting to run new task attempt: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,304 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,304 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,304 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:53,304 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:53,304 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:53,304 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:53,305 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:53,305 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown, VertexName: BEB67DFC5DAB4C0D930D58116C1C92B7, VertexParallelism: 1, TaskAttemptID:attempt_1463517953104_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C9009DA06C044CBCABD8DC9B87438BBD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:53,305 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:53,310 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953104_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:53,312 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,312 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:53,312 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,313 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C9009DA06C044CBCABD8DC9B87438BBD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:53,313 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:53,313 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,314 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,314 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,316 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:53,316 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,316 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,316 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:53,316 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:53,317 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,317 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:53,318 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:53,318 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,319 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,327 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BEB67DFC5DAB4C0D930D58116C1C92B7, ordinal: 0
2016-05-17 20:45:53,327 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEB67DFC5DAB4C0D930D58116C1C92B7, mem on start (mb), free: 258, total: 541, max: 672
2016-05-17 20:45:53,327 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,327 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:53,327 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:53,327 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:53,327 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:53,328 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:53,328 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:53,328 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:53,328 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:53,328 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:53,328 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C9009DA06C044CBCABD8DC9B87438BBD, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:53,328 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F4FD1F2D4E2B402BA1A24AC7D23055F6, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:53,329 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:53,329 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C9009DA06C044CBCABD8DC9B87438BBD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:53,329 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:53,329 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:53,329 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,329 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:53,329 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:53,330 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,330 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:53,330 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1015fc27
2016-05-17 20:45:53,331 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1015fc27
2016-05-17 20:45:53,332 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:53,332 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:53,333 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:53,333 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,333 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:-1]], id: 33A04523F6CE49278B94AF86B1734C29
2016-05-17 20:45:53,333 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:-1]] 33A04523F6CE49278B94AF86B1734C29
2016-05-17 20:45:53,335 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3cfe7c4c; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@679280dc; comparator=cascading.tuple.hadoop.util.TupleComparator@434a20a3; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:53,335 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:53,335 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:53,335 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:53,359 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:53,359 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:53,359 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:53,359 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:53,359 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:53,360 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:53,360 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:53,360 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BEB67DFC5DAB4C0D930D58116C1C92B7, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:53,362 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,362 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEB67DFC5DAB4C0D930D58116C1C92B7, mem on close (mb), free: 322, total: 554, max: 672
2016-05-17 20:45:53,363 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:53,363 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:53,363 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:53,363 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:53,365 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:53 Completed running task attempt: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,368 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:53,369 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953104_0001_1_00, eventCount=1
2016-05-17 20:45:53,370 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0, startTime=1463517953302, finishTime=1463517953370, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953360, Process_Duration=2, Process_End_Time=1463517953362, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:53,370 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:53,370 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:53,370 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,371 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:53,371 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_FINISHED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, taskId=task_1463517953104_0001_1_00_000000, startTime=1463517953302, finishTime=1463517953371, timeTaken=69, status=SUCCEEDED, successfulAttemptID=attempt_1463517953104_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953360, Process_Duration=2, Process_End_Time=1463517953362, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:53,371 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:53,372 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] : 1
2016-05-17 20:45:53,372 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,373 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:53,373 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_FINISHED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, vertexId=vertex_1463517953104_0001_1_00, initRequestedTime=1463517953281, initedTime=1463517953291, startRequestedTime=1463517953291, startedTime=1463517953291, finishTime=1463517953372, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953360, Process_Duration=2, Process_End_Time=1463517953362, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517953302, firstTasksToStart=[ task_1463517953104_0001_1_00_000000 ], lastTaskFinishTime=1463517953370, lastTasksToFinish=[ task_1463517953104_0001_1_00_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953104_0001_1_00_000000 ], longestDurationTasks=[ task_1463517953104_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:53,373 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:53,374 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] attempt: attempt_1463517953104_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:53,374 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,374 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,374 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:53,374 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_STARTED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, taskId=task_1463517953104_0001_1_01_000000, scheduledTime=1463517953374, launchTime=1463517953374
2016-05-17 20:45:53,375 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:53,375 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,375 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown, VertexName: F4FD1F2D4E2B402BA1A24AC7D23055F6, VertexParallelism: 1, TaskAttemptID:attempt_1463517953104_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=24E8A8ECAD434F3F811665337F82343F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:53,373 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:53,375 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:53,376 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:53,378 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:53,378 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517953360, Process_Duration=2, Process_End_Time=1463517953362, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:53,378 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:53,378 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:53,379 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,379 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953104_0001_00_000001
2016-05-17 20:45:53,379 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:53,379 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517953104_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:53,379 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953104_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:53,379 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517953104_0001_00_000001, Exiting
2016-05-17 20:45:53,380 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953104_0001_00_000001
2016-05-17 20:45:53,380 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953104_0001_00_000001
2016-05-17 20:45:53,379 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953104_0001_00_000001, stoppedTime=1463517953379, exitStatus=0
2016-05-17 20:45:53,381 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517953104_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:53,381 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:53,382 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517953104_0001_00_000001 completed successfully
2016-05-17 20:45:53,382 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953104_0001_00_000002
2016-05-17 20:45:53,382 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:53,382 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:53,382 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517953104_0001_00_000001
2016-05-17 20:45:53,383 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517953104_0001_00_000002
2016-05-17 20:45:53,384 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:53,385 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:53,386 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:53,386 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953104_0001_00_000002, launchTime=1463517953386
2016-05-17 20:45:53,386 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953104_0001_00_000002
2016-05-17 20:45:53,387 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953104_0001_1_01_000000_0] to container: [container_1463517953104_0001_00_000002]
2016-05-17 20:45:53,387 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:53,387 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953104_0001_00_000002 given task: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,387 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953104_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,388 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953104_0001_1_01_000000_0] started. Is using containerId: [container_1463517953104_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:53 Starting to run new task attempt: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,388 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0, startTime=1463517953387, containerId=container_1463517953104_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953104_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:53,388 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,388 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:53,388 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:53,388 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,388 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:53,388 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:53,389 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:53,389 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:53,389 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:53,389 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown, VertexName: F4FD1F2D4E2B402BA1A24AC7D23055F6, VertexParallelism: 1, TaskAttemptID:attempt_1463517953104_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=24E8A8ECAD434F3F811665337F82343F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:53,388 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:53,389 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:53,389 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,389 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:53,390 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,395 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953104_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:53,395 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,396 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=24E8A8ECAD434F3F811665337F82343F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:53,397 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:53,397 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 24E8A8ECAD434F3F811665337F82343F
2016-05-17 20:45:53,397 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:53,397 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 24E8A8ECAD434F3F811665337F82343F
2016-05-17 20:45:53,397 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:53,398 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:53,398 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,398 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:53,399 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:53,399 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:53,400 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,400 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,406 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F4FD1F2D4E2B402BA1A24AC7D23055F6, ordinal: 1
2016-05-17 20:45:53,406 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4FD1F2D4E2B402BA1A24AC7D23055F6, mem on start (mb), free: 315, total: 554, max: 672
2016-05-17 20:45:53,407 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,407 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:53,407 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:53,408 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:53,409 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 24E8A8ECAD434F3F811665337F82343F
2016-05-17 20:45:53,409 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:53,409 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:53,409 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:53,409 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:53,409 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:53,409 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:53,409 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:53,410 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 24E8A8ECAD434F3F811665337F82343F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:53,410 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BEB67DFC5DAB4C0D930D58116C1C92B7, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:53,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:53,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BEB67DFC5DAB4C0D930D58116C1C92B7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:53,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:53,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:53,411 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:53,411 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:53,411 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BEB67DFC5DAB4C0D930D58116C1C92B7 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:53,411 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:53,412 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:53,412 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:53,413 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,413 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:53,413 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,413 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517953104_0001_1_00_000000_0_10049, runDuration: 50365, ]
2016-05-17 20:45:53,415 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517953104_0001_1_00_000000_0_10049] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517953415, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 20:45:53,415 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,415 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:53,415 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BEB67DFC5DAB4C0D930D58116C1C92B7] #8 in 1ms
2016-05-17 20:45:53,416 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BEB67DFC5DAB4C0D930D58116C1C92B7] #8, status:false, isInterrupted:false
2016-05-17 20:45:53,416 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:-1]]
2016-05-17 20:45:53,416 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:53,417 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:-1]] streamed: true, id: 33A04523F6CE49278B94AF86B1734C29
2016-05-17 20:45:53,417 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:53,417 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"], id: 24E8A8ECAD434F3F811665337F82343F
2016-05-17 20:45:53,417 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:53,417 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:53,417 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-17 20:45:53,418 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:53,418 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,418 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:53,418 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:-1]] 33A04523F6CE49278B94AF86B1734C29, for 1 inputs
2016-05-17 20:45:53,418 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F4FD1F2D4E2B402BA1A24AC7D23055F6, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:53,420 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@53360449; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7f537f4c
2016-05-17 20:45:53,422 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:53,422 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:53,422 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,423 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517953104_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:53,423 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,424 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4FD1F2D4E2B402BA1A24AC7D23055F6, mem on close (mb), free: 313, total: 554, max: 672
2016-05-17 20:45:53,424 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,424 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:53,425 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:53,425 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:53,425 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:53,425 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:53 Completed running task attempt: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,426 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:53,427 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517953419, Process_Duration=4, Process_End_Time=1463517953423, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 20:45:53,427 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:53,428 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:53,428 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,428 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953104_0001_00_000002
2016-05-17 20:45:53,427 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0, startTime=1463517953387, finishTime=1463517953427, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953419, Process_Duration=4, Process_End_Time=1463517953423, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:53,429 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:53,429 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:53,429 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,429 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:53,430 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_FINISHED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, taskId=task_1463517953104_0001_1_01_000000, startTime=1463517953387, finishTime=1463517953429, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463517953104_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953419, Process_Duration=4, Process_End_Time=1463517953423, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:53,430 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:53,430 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] : 1
2016-05-17 20:45:53,430 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,431 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:53,431 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_FINISHED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, vertexId=vertex_1463517953104_0001_1_01, initRequestedTime=1463517953282, initedTime=1463517953289, startRequestedTime=1463517953293, startedTime=1463517953293, finishTime=1463517953430, timeTaken=137, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953419, Process_Duration=4, Process_End_Time=1463517953423, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463517953387, firstTasksToStart=[ task_1463517953104_0001_1_01_000000 ], lastTaskFinishTime=1463517953427, lastTasksToFinish=[ task_1463517953104_0001_1_01_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953104_0001_1_01_000000 ], longestDurationTasks=[ task_1463517953104_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:53,431 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:53,431 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:53,432 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953104_0001_00_000002, stoppedTime=1463517953432, exitStatus=0
2016-05-17 20:45:53,432 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:53,432 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:53,432 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517953104_0001_1
2016-05-17 20:45:53,433 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517953104_0001_1
2016-05-17 20:45:53,433 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,433 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 24E8A8ECAD434F3F811665337F82343F for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,434 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:53,439 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517953104_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:53,439 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953104_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:53,439 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517953104_0001_00_000002, Exiting
2016-05-17 20:45:53,440 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953104_0001_00_000002
2016-05-17 20:45:53,440 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953104_0001_00_000002
2016-05-17 20:45:53,440 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517953104_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:53,440 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517953104_0001_00_000002 completed successfully
2016-05-17 20:45:53,442 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517953104_0001_1, queueSize=0
2016-05-17 20:45:53,442 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:53,443 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517953104_0001_1, startTime=1463517953280, finishTime=1463517953440, timeTaken=160, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=189, FILE_BYTES_WRITTEN=159, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1161822208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035906779, Process_Duration=6, Process_End_Time=2927035906785, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-17 20:45:53,443 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517953104_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:53,444 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953104_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:53,444 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:53,444 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517953104_0001_00_000002
2016-05-17 20:45:53 Completed Dag: dag_1463517953104_0001_1
2016-05-17 20:45:53,446 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517953104_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:53,447 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517953104_0001_1_post
2016-05-17 20:45:53,447 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,447 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:53,451 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463517953104_0001
2016-05-17 20:45:53,453 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:53,453 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:53,453 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:53,453 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:53,453 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:53,453 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:53,454 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:53,454 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:53,454 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:53,454 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:53,454 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:53,455 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:53,455 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:53,459 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001
2016-05-17 20:45:53,460 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown/_temporary
2016-05-17 20:45:53,460 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.373
2016-05-17 20:45:53,515 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-17 20:45:53,543 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:53,543 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:53,588 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 20:45:53,606 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:53,606 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:53,615 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.049
2016-05-17 20:45:53,615 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:53,616 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:53,616 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:53,639 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-05-17 20:45:53,646 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 20:45:53,652 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-17 20:45:53,753 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-17 20:45:53,754 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:53,754 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:53,754 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-17 20:45:53,755 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  parallel execution of steps is enabled: false
2016-05-17 20:45:53,755 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  executing total steps: 1
2016-05-17 20:45:53,755 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  allocating management threads: 1
2016-05-17 20:45:53,756 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53,756 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:53,768 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-17 20:45:53,772 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:53,773 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:53,773 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:53,783 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001 doesn't exist and is created
2016-05-17 20:45:53,799 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:53,799 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:53,800 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001
2016-05-17 20:45:53,804 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517953773_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:53,819 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:53,820 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:53,823 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:53,823 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:53,823 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001/localmode-log-dir
2016-05-17 20:45:53,824 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001/localmode-log-dir/history.txt.appattempt_1463517953773_0001_000000, maxErrors=10
2016-05-17 20:45:53,824 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:53,825 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517953773_0001_000000, appSubmitTime=1463517953804, launchTime=1463517953804
2016-05-17 20:45:53,830 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:53,833 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:53,833 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:53,834 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:53,838 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56526
2016-05-17 20:45:53,840 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:53,841 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517953773_0001_000000, startTime=1463517953841
2016-05-17 20:45:53,842 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:53,842 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:53,899 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:53,900 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:53,900 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463517953773_0001, dagName=[B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53,919 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53,920 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517953773_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001/localmode-log-dir/dag_1463517953773_0001_1.dot
2016-05-17 20:45:53,921 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001/localmode-log-dir/dag_1463517953773_0001_1-tez-dag.pb.txt
2016-05-17 20:45:53,931 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517953773_0001_1
2016-05-17 20:45:53,932 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,932 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53 Running Dag: dag_1463517953773_0001_1
2016-05-17 20:45:53,946 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517953773_0001_1, submitTime=1463517953919
2016-05-17 20:45:53,946 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:53,947 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:53,947 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:53,948 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517953773_0001_1, initTime=1463517953946
2016-05-17 20:45:53,948 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:53,948 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953773_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:53,949 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_STARTED]: dagID=dag_1463517953773_0001_1, startTime=1463517953949
2016-05-17 20:45:53,949 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:53,949 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463517953773_0001, dagName=[B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53,950 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953773_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:53,950 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F7C9F8290A98443ABC86347D3CDCB5E3 : {6837B65D2EA7478B8D296BC295D4907B={InputName=6837B65D2EA7478B8D296BC295D4907B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:53,950 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6837B65D2EA7478B8D296BC295D4907B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:53,950 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,950 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,950 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,951 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]. Starting root input initializers: 1
2016-05-17 20:45:53,951 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:53,951 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B37A4A0444944283B5C65E2C19FC35BA : {EA413A1D8AA244E1B9EF6B102662BC59={InputName=EA413A1D8AA244E1B9EF6B102662BC59}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:53,951 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EA413A1D8AA244E1B9EF6B102662BC59, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:53,951 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,952 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,952 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,952 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]. Starting root input initializers: 1
2016-05-17 20:45:53,951 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:53,953 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6837B65D2EA7478B8D296BC295D4907B on vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,954 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:53,955 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EA413A1D8AA244E1B9EF6B102662BC59 on vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,955 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,956 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:53,957 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6837B65D2EA7478B8D296BC295D4907B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:53,957 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:53,958 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:53,958 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:53,958 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,959 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,959 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,960 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=763782EE94B642ECB31B619719C37B70, vertexId=vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:53,960 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=763782EE94B642ECB31B619719C37B70, vertexId=vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,962 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:53,962 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=763782EE94B642ECB31B619719C37B70, vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:53,962 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EA413A1D8AA244E1B9EF6B102662BC59 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:53,963 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=763782EE94B642ECB31B619719C37B70, vertexId=vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,965 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:53,965 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6837B65D2EA7478B8D296BC295D4907B on vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,965 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:53,966 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,966 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, vertexId=vertex_1463517953773_0001_1_02, initRequestedTime=1463517953955, initedTime=1463517953965, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:53,966 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:53,967 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:53,967 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,969 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6837B65D2EA7478B8D296BC295D4907B=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:53,969 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] parallelism set to 1
2016-05-17 20:45:53,970 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,970 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953773_0001_1_00, eventCount=1
2016-05-17 20:45:53,967 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463517953773_0001, with dag id: dag_1463517953773_0001_1
2016-05-17 20:45:53,970 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:53,970 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, vertexId=vertex_1463517953773_0001_1_00, initRequestedTime=1463517953950, initedTime=1463517953970, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:53,972 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] with distanceFromRoot: 0
2016-05-17 20:45:53,972 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:53,970 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,972 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:53,972 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,972 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EA413A1D8AA244E1B9EF6B102662BC59 on vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,973 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,973 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953773_0001_1_00, startRequestedTime=1463517953972, startedTime=1463517953972
2016-05-17 20:45:53,975 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,975 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:53,975 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_STARTED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, taskId=task_1463517953773_0001_1_00_000000, scheduledTime=1463517953975, launchTime=1463517953975
2016-05-17 20:45:53,976 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:53,976 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,976 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EA413A1D8AA244E1B9EF6B102662BC59=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:53,977 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] parallelism set to 1
2016-05-17 20:45:53,977 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,977 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953773_0001_1_01, eventCount=1
2016-05-17 20:45:53,978 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, vertexId=vertex_1463517953773_0001_1_01, initRequestedTime=1463517953951, initedTime=1463517953978, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:53,978 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] with distanceFromRoot: 0
2016-05-17 20:45:53,978 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,980 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:53,981 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517953773_0001_1_00 for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] numStartedSources: 1 numSources: 2
2016-05-17 20:45:53,981 INFO  impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] numStartedSources: 1 numSources: 2
2016-05-17 20:45:53,981 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,982 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:53,982 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,982 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953773_0001_1_01, startRequestedTime=1463517953982, startedTime=1463517953982
2016-05-17 20:45:53,982 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,985 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,986 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: F7C9F8290A98443ABC86347D3CDCB5E3, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6837B65D2EA7478B8D296BC295D4907B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:53,986 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:53,986 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:53,987 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_STARTED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, taskId=task_1463517953773_0001_1_01_000000, scheduledTime=1463517953986, launchTime=1463517953986
2016-05-17 20:45:53,987 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:53,987 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517953773_0001_1_01 for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] numStartedSources: 2 numSources: 2
2016-05-17 20:45:53,987 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] with distanceFromRoot: 1
2016-05-17 20:45:53,987 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:53,988 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:53,987 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,989 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B37A4A0444944283B5C65E2C19FC35BA in vertex: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:53,990 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F7C9F8290A98443ABC86347D3CDCB5E3 in vertex: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:53,990 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FEBA4EBF8BDB40D78748DD504B2E84DE with 2 source tasks and 1 pending tasks
2016-05-17 20:45:53,990 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,990 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953773_0001_1_02, startRequestedTime=1463517953989, startedTime=1463517953989
2016-05-17 20:45:53,991 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,991 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:53,992 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:53,993 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953773_0001_00_000001
2016-05-17 20:45:53,993 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:53,993 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: B37A4A0444944283B5C65E2C19FC35BA, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EA413A1D8AA244E1B9EF6B102662BC59, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:53,993 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:53,994 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:53,995 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517953773_0001_00_000001
2016-05-17 20:45:53,995 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:53,996 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:53,997 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953773_0001_00_000001, launchTime=1463517953996
2016-05-17 20:45:53,997 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000001
2016-05-17 20:45:53,998 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953773_0001_00_000002
2016-05-17 20:45:53,998 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:53,999 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:54,000 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517953773_0001_00_000002
2016-05-17 20:45:54,000 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:54,003 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:54,004 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:54,024 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953773_0001_00_000002, launchTime=1463517954002
2016-05-17 20:45:54,024 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953773_0001_1_00_000000_0] to container: [container_1463517953773_0001_00_000001]
2016-05-17 20:45:54,024 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:54,024 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953773_0001_00_000001 given task: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,025 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000001: 27 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,025 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953773_0001_1_00_000000_0] started. Is using containerId: [container_1463517953773_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:54,026 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0, startTime=1463517954025, containerId=container_1463517953773_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953773_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:54,026 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:54,026 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:54 Starting to run new task attempt: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,027 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,027 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,027 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:54,027 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:54,027 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:54,027 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:54,027 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:54,027 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:54,027 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: F7C9F8290A98443ABC86347D3CDCB5E3, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6837B65D2EA7478B8D296BC295D4907B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:54,028 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:54,032 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000002
2016-05-17 20:45:54,038 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,039 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:54,039 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,040 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6837B65D2EA7478B8D296BC295D4907B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:54,040 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:54,040 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,040 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,041 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,060 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953773_0001_1_01_000000_0] to container: [container_1463517953773_0001_00_000002]
2016-05-17 20:45:54,060 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:54,060 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953773_0001_1_01_000000_0] started. Is using containerId: [container_1463517953773_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:54,061 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0, startTime=1463517954060, containerId=container_1463517953773_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953773_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:54,061 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:54,061 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:54,061 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953773_0001_00_000002 given task: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,061 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,061 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:54 Starting to run new task attempt: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,062 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,062 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,062 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:54,062 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:54,062 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:54,062 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:54,063 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:54,063 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: B37A4A0444944283B5C65E2C19FC35BA, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EA413A1D8AA244E1B9EF6B102662BC59, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:54,063 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:54,063 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:54,064 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:54,064 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:54,064 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,066 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:54,066 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,068 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,069 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:54,068 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953773_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:54,068 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,072 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:54,072 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,072 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,072 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EA413A1D8AA244E1B9EF6B102662BC59, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:54,073 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953773_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:54,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:54,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,074 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,075 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,076 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:54,076 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:54,076 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,078 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:54,078 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,080 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:54,082 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:54,082 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,083 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,087 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F7C9F8290A98443ABC86347D3CDCB5E3, ordinal: 1
2016-05-17 20:45:54,088 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7C9F8290A98443ABC86347D3CDCB5E3, mem on start (mb), free: 416, total: 553, max: 672
2016-05-17 20:45:54,088 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:54,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:54,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:54,089 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B37A4A0444944283B5C65E2C19FC35BA, ordinal: 0
2016-05-17 20:45:54,090 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B37A4A0444944283B5C65E2C19FC35BA, mem on start (mb), free: 416, total: 553, max: 672
2016-05-17 20:45:54,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,090 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:54,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:54,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:54,091 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:54,091 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:54,091 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:54,091 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:54,092 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:54,092 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:54,092 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EA413A1D8AA244E1B9EF6B102662BC59, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:54,092 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FEBA4EBF8BDB40D78748DD504B2E84DE, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:54,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:54,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EA413A1D8AA244E1B9EF6B102662BC59 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:54,092 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:54,093 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:54,090 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:54,093 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:54,093 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:54,093 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:54,093 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:54,093 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:54,094 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6837B65D2EA7478B8D296BC295D4907B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:54,094 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FEBA4EBF8BDB40D78748DD504B2E84DE, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:54,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:54,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,094 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:54,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6837B65D2EA7478B8D296BC295D4907B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:54,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,095 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:54,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:54,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:54,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:54,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:54,096 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,097 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:54,098 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@169cc168
2016-05-17 20:45:54,099 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@169cc168
2016-05-17 20:45:54,102 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:54,102 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:54,102 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,101 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:54,103 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,104 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,104 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,106 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:54,107 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,107 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,107 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,110 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@27c90d11; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@78eb7f; comparator=cascading.tuple.hadoop.util.TupleComparator@1cdd50ca; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:54,110 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2acdf17; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@236a6060; comparator=cascading.tuple.hadoop.util.TupleComparator@5e30c94; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:54,114 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:54,114 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:54,114 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:54,115 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:54,115 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:54,115 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:54,113 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:54,117 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@b2e359a
2016-05-17 20:45:54,118 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@b2e359a
2016-05-17 20:45:54,118 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:54,118 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:54,131 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:54,131 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:54,132 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:54,132 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:54,132 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:54,131 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:54,133 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:54,133 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:54,133 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:54,133 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:54,134 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:54,134 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F7C9F8290A98443ABC86347D3CDCB5E3, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:54,134 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:54,136 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,137 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7C9F8290A98443ABC86347D3CDCB5E3, mem on close (mb), free: 213, total: 553, max: 672
2016-05-17 20:45:54,137 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:54,137 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:54,137 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:54,137 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:54,138 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:54,138 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:54,138 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:54,139 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:54,139 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B37A4A0444944283B5C65E2C19FC35BA, all 1 inputs ready in: 00:00:00.001
2016-05-17 20:45:54,142 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,143 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B37A4A0444944283B5C65E2C19FC35BA, mem on close (mb), free: 213, total: 553, max: 672
2016-05-17 20:45:54,143 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:54,144 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:54,144 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:54,144 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:54,491 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:54,491 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:54 Completed running task attempt: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,495 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:54 Completed running task attempt: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,497 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:54,498 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=358, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517954134, Process_Duration=2, Process_End_Time=1463517954136, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:54,498 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953773_0001_1_00, eventCount=1
2016-05-17 20:45:54,499 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:54,499 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,499 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953773_0001_1_01, eventCount=1
2016-05-17 20:45:54,499 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000001
2016-05-17 20:45:54,499 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:54,500 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0, startTime=1463517954025, finishTime=1463517954499, timeTaken=474, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=358, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954134, Process_Duration=2, Process_End_Time=1463517954136, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:54,500 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:54,501 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0, startTime=1463517954060, finishTime=1463517954500, timeTaken=440, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=343, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954139, Process_Duration=3, Process_End_Time=1463517954142, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:54,501 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:54,501 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,501 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_FINISHED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, taskId=task_1463517953773_0001_1_00_000000, startTime=1463517954025, finishTime=1463517954501, timeTaken=476, status=SUCCEEDED, successfulAttemptID=attempt_1463517953773_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=358, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954134, Process_Duration=2, Process_End_Time=1463517954136, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:54,501 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:54,501 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:54,502 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:54,502 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:54,502 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,504 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:54,504 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=343, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517954139, Process_Duration=3, Process_End_Time=1463517954142, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:54,504 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_FINISHED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, taskId=task_1463517953773_0001_1_01_000000, startTime=1463517954060, finishTime=1463517954503, timeTaken=443, status=SUCCEEDED, successfulAttemptID=attempt_1463517953773_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=343, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954139, Process_Duration=3, Process_End_Time=1463517954142, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:54,504 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:54,504 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:54,504 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,504 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:54,505 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] : 1
2016-05-17 20:45:54,505 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:54,505 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000002
2016-05-17 20:45:54,506 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:54,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_FINISHED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, vertexId=vertex_1463517953773_0001_1_00, initRequestedTime=1463517953950, initedTime=1463517953970, startRequestedTime=1463517953972, startedTime=1463517953972, finishTime=1463517954505, timeTaken=533, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=358, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954134, Process_Duration=2, Process_End_Time=1463517954136, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517954025, firstTasksToStart=[ task_1463517953773_0001_1_00_000000 ], lastTaskFinishTime=1463517954499, lastTasksToFinish=[ task_1463517953773_0001_1_00_000000 ], minTaskDuration=474, maxTaskDuration=474, avgTaskDuration=474.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953773_0001_1_00_000000 ], longestDurationTasks=[ task_1463517953773_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:54,507 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:54,507 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] : 1
2016-05-17 20:45:54,507 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:54,509 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:54,509 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_FINISHED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, vertexId=vertex_1463517953773_0001_1_01, initRequestedTime=1463517953951, initedTime=1463517953978, startRequestedTime=1463517953982, startedTime=1463517953982, finishTime=1463517954507, timeTaken=525, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=343, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954139, Process_Duration=3, Process_End_Time=1463517954142, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517954060, firstTasksToStart=[ task_1463517953773_0001_1_01_000000 ], lastTaskFinishTime=1463517954500, lastTasksToFinish=[ task_1463517953773_0001_1_01_000000 ], minTaskDuration=440, maxTaskDuration=440, avgTaskDuration=440.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953773_0001_1_01_000000 ], longestDurationTasks=[ task_1463517953773_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:54,509 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:54,509 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:54,509 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:54,510 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517953773_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:54,510 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:54,510 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517953773_0001_00_000001, Exiting
2016-05-17 20:45:54,510 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953773_0001_00_000001
2016-05-17 20:45:54,510 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953773_0001_00_000001
2016-05-17 20:45:54,511 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953773_0001_00_000001, stoppedTime=1463517954510, exitStatus=0
2016-05-17 20:45:54,512 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:54,512 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953773_0001_00_000002, stoppedTime=1463517954512, exitStatus=0
2016-05-17 20:45:54,512 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:54,511 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:54,513 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:54,514 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517953773_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:54,516 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:54,516 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517953773_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:54,514 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] attempt: attempt_1463517953773_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:54,516 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517953773_0001_00_000002, Exiting
2016-05-17 20:45:54,517 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953773_0001_00_000002
2016-05-17 20:45:54,517 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953773_0001_00_000002
2016-05-17 20:45:54,517 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:54,518 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] attempt: attempt_1463517953773_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:54,518 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,518 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:54,518 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:54,518 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517953773_0001_00_000001
2016-05-17 20:45:54,518 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517953773_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 20:45:54,519 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:54,519 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517953773_0001_00_000001 completed successfully
2016-05-17 20:45:54,518 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_STARTED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, taskId=task_1463517953773_0001_1_02_000000, scheduledTime=1463517954518, launchTime=1463517954518
2016-05-17 20:45:54,519 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517953773_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:54,519 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:54,520 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517953773_0001_00_000002 completed successfully
2016-05-17 20:45:54,520 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:54,521 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:54,521 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:54,521 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:54,521 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:54,521 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: FEBA4EBF8BDB40D78748DD504B2E84DE, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B37A4A0444944283B5C65E2C19FC35BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=F7C9F8290A98443ABC86347D3CDCB5E3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=763782EE94B642ECB31B619719C37B70, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:54,521 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:54,522 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:54,522 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953773_0001_00_000003
2016-05-17 20:45:54,522 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:54,524 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517953773_0001_00_000003
2016-05-17 20:45:54,525 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:54,527 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:54,527 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:54,528 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953773_0001_00_000003, launchTime=1463517954527
2016-05-17 20:45:54,528 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,529 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953773_0001_1_02_000000_0] to container: [container_1463517953773_0001_00_000003]
2016-05-17 20:45:54,529 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:54,530 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953773_0001_1_02_000000_0] started. Is using containerId: [container_1463517953773_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 20:45:54,530 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0, startTime=1463517954530, containerId=container_1463517953773_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953773_0001_00_000003/teamcity, completedLogs=
2016-05-17 20:45:54,530 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:54,530 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:54,531 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:54,532 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953773_0001_00_000003 given task: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,532 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54 Starting to run new task attempt: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,533 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,533 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,533 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:54,533 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:54,533 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:54,533 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:54,533 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:54,533 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: FEBA4EBF8BDB40D78748DD504B2E84DE, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B37A4A0444944283B5C65E2C19FC35BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=F7C9F8290A98443ABC86347D3CDCB5E3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=763782EE94B642ECB31B619719C37B70, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:54,534 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:54,535 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,539 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:54,539 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,540 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953773_0001_1_02_000000_0 sent events: (0-2)
2016-05-17 20:45:54,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B37A4A0444944283B5C65E2C19FC35BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:54,541 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=763782EE94B642ECB31B619719C37B70, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:54,541 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:54,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F7C9F8290A98443ABC86347D3CDCB5E3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:54,541 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:54,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:54,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 763782EE94B642ECB31B619719C37B70
2016-05-17 20:45:54,542 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:54,542 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 763782EE94B642ECB31B619719C37B70
2016-05-17 20:45:54,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,549 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,549 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:54,554 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:54,554 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:54,554 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,558 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:54,558 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:54,558 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,561 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,561 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FEBA4EBF8BDB40D78748DD504B2E84DE, ordinal: 2
2016-05-17 20:45:54,561 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:54,562 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 763782EE94B642ECB31B619719C37B70
2016-05-17 20:45:54,562 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FEBA4EBF8BDB40D78748DD504B2E84DE, mem on start (mb), free: 424, total: 652, max: 672
2016-05-17 20:45:54,562 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,563 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-05-17 20:45:54,563 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:54,563 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:54,563 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:54,564 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:54,564 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:54,564 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:54,564 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:54,565 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:54,565 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-17 20:45:54,565 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-05-17 20:45:54,566 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 763782EE94B642ECB31B619719C37B70, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:54,566 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B37A4A0444944283B5C65E2C19FC35BA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 20:45:54,566 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F7C9F8290A98443ABC86347D3CDCB5E3, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 20:45:54,566 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:54,567 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,567 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B37A4A0444944283B5C65E2C19FC35BA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:54,567 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:54,568 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:54,568 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B37A4A0444944283B5C65E2C19FC35BA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:54,568 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,568 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 20:45:54,569 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:54,570 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:54,569 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:54,569 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F7C9F8290A98443ABC86347D3CDCB5E3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:54,570 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:54,573 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F7C9F8290A98443ABC86347D3CDCB5E3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:54,573 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 20:45:54,574 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,575 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:54,576 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:54,575 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-05-17 20:45:54,578 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-05-17 20:45:54,578 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:54,578 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,578 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:54,579 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,580 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517953773_0001_1_00_000000_0_10055, runDuration: 429323, ]
2016-05-17 20:45:54,581 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517953773_0001_1_00_000000_0_10055] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517954581, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 20:45:54,581 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,581 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:54,581 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [F7C9F8290A98443ABC86347D3CDCB5E3] #10 in 1ms
2016-05-17 20:45:54,582 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F7C9F8290A98443ABC86347D3CDCB5E3] #10, status:false, isInterrupted:false
2016-05-17 20:45:54,583 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:54,584 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:54,584 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:54,585 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:54,585 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:54,585 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,585 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2016-05-17 20:45:54,585 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517953773_0001_1_01_000000_0_10058, runDuration: 417847, ]
2016-05-17 20:45:54,585 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:54,586 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] streamed: true, id: A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,586 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"], id: 763782EE94B642ECB31B619719C37B70
2016-05-17 20:45:54,586 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-17 20:45:54,586 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] A4FADDD631CF4E2793C0FF65D4349180, for 2 inputs
2016-05-17 20:45:54,587 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] A4FADDD631CF4E2793C0FF65D4349180, for 2 inputs
2016-05-17 20:45:54,587 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517953773_0001_1_01_000000_0_10058] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517954587, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 20:45:54,587 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,587 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:54,588 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B37A4A0444944283B5C65E2C19FC35BA] #9 in 2ms
2016-05-17 20:45:54,590 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 20:45:54,590 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 20:45:54,593 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B37A4A0444944283B5C65E2C19FC35BA] #9, status:false, isInterrupted:false
2016-05-17 20:45:54,594 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:54,595 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:54,595 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:54,595 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:54,596 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:54,596 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FEBA4EBF8BDB40D78748DD504B2E84DE, all 2 inputs ready in: 00:00:00.006
2016-05-17 20:45:54,599 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7a0d63a1; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5081da1
2016-05-17 20:45:54,599 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,600 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:54,602 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@57d1379b; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@6899ce27
2016-05-17 20:45:54,604 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:54,605 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:54,606 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,606 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517953773_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 20:45:54,607 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,608 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FEBA4EBF8BDB40D78748DD504B2E84DE, mem on close (mb), free: 420, total: 652, max: 672
2016-05-17 20:45:54,608 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,609 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:54,610 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:54,610 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:54,611 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:54,611 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:54,612 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,612 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:54,613 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:54,613 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:54,614 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:54,614 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:54 Completed running task attempt: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,616 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:54,618 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517954596, Process_Duration=11, Process_End_Time=1463517954607, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 20:45:54,618 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:54,618 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0, startTime=1463517954530, finishTime=1463517954618, timeTaken=88, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954596, Process_Duration=11, Process_End_Time=1463517954607, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:54,619 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:54,619 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,620 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:54,619 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:54,620 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_FINISHED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, taskId=task_1463517953773_0001_1_02_000000, startTime=1463517954530, finishTime=1463517954620, timeTaken=90, status=SUCCEEDED, successfulAttemptID=attempt_1463517953773_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954596, Process_Duration=11, Process_End_Time=1463517954607, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:54,621 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:54,621 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] : 1
2016-05-17 20:45:54,621 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:54,622 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:54,622 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_FINISHED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, vertexId=vertex_1463517953773_0001_1_02, initRequestedTime=1463517953955, initedTime=1463517953965, startRequestedTime=1463517953989, startedTime=1463517953989, finishTime=1463517954621, timeTaken=632, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954596, Process_Duration=11, Process_End_Time=1463517954607, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463517954530, firstTasksToStart=[ task_1463517953773_0001_1_02_000000 ], lastTaskFinishTime=1463517954618, lastTasksToFinish=[ task_1463517953773_0001_1_02_000000 ], minTaskDuration=88, maxTaskDuration=88, avgTaskDuration=88.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953773_0001_1_02_000000 ], longestDurationTasks=[ task_1463517953773_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:54,623 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:54,623 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:54,623 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517953773_0001_1
2016-05-17 20:45:54,623 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517953773_0001_1
2016-05-17 20:45:54,623 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 763782EE94B642ECB31B619719C37B70 for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:54,619 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:54,629 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,634 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:54,634 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:54,637 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517953773_0001_1, queueSize=0
2016-05-17 20:45:54,637 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:54,638 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517953773_0001_1, startTime=1463517953949, finishTime=1463517954634, timeTaken=685, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=502, FILE_BYTES_WRITTEN=2632, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=701, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=2052587520, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390553862869, Process_Duration=16, Process_End_Time=4390553862885, Read_Duration=5, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-17 20:45:54,638 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517953773_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:54,638 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953773_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:54,639 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:54,642 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953773_0001_00_000003, stoppedTime=1463517954642, exitStatus=0
2016-05-17 20:45:54,642 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:54,642 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:54 Completed Dag: dag_1463517953773_0001_1
2016-05-17 20:45:54,643 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517953773_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:54,644 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517953773_0001_1_post
2016-05-17 20:45:54,644 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,644 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:54,644 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517953773_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-17 20:45:54,645 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:54,645 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463517953773_0001
2016-05-17 20:45:54,647 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:54,647 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:54,647 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:54,647 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:54,647 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:54,648 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:54,648 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:54,648 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:54,648 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:54,649 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:54,649 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:54,649 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:54,649 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:54,649 INFO  task.TezChild (TezChild.java:run(198)) - Interrupted while waiting for new work for container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,649 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,649 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,655 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001
2016-05-17 20:45:54,656 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup/_temporary
2016-05-17 20:45:54,656 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  completed in: 00:00.902
2016-05-17 20:45:54,729 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-17 20:45:54,751 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:54,751 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:54,786 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 20:45:54,790 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 20:45:54,790 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:54,790 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:54,790 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:54,804 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:54,804 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-17 20:45:54,812 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:54,812 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:54,813 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-17 20:45:54,813 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:54,814 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:54,814 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:54,815 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...tform/testReplace/replace
2016-05-17 20:45:54,815 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:54,827 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:54,831 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:54,832 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:54,832 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:54,841 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001 doesn't exist and is created
2016-05-17 20:45:54,855 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:54,855 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:54,856 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001
2016-05-17 20:45:54,858 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517954832_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:54,869 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:54,869 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:54,871 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:54,871 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:54,871 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001/localmode-log-dir
2016-05-17 20:45:54,871 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001/localmode-log-dir/history.txt.appattempt_1463517954832_0001_000000, maxErrors=10
2016-05-17 20:45:54,871 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:54,872 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517954832_0001_000000, appSubmitTime=1463517954858, launchTime=1463517954858
2016-05-17 20:45:54,874 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:54,876 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:54,878 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:54,878 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:54,880 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43863
2016-05-17 20:45:54,881 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:54,884 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517954832_0001_000000, startTime=1463517954884
2016-05-17 20:45:54,884 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:54,884 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:54,955 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:54,956 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:54,956 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463517954832_0001, dagName=[7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace
2016-05-17 20:45:54,979 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace
2016-05-17 20:45:54,980 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517954832_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001/localmode-log-dir/dag_1463517954832_0001_1.dot
2016-05-17 20:45:54,981 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001/localmode-log-dir/dag_1463517954832_0001_1-tez-dag.pb.txt
2016-05-17 20:45:54,989 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517954832_0001_1
2016-05-17 20:45:54,989 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,990 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace
2016-05-17 20:45:54 Running Dag: dag_1463517954832_0001_1
2016-05-17 20:45:55,002 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517954832_0001_1, submitTime=1463517954979
2016-05-17 20:45:55,003 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:55,003 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E80C71241AB144E3807B712750E7D22B
2016-05-17 20:45:55,003 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:55,004 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:55,005 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517954832_0001_1, initTime=1463517955003
2016-05-17 20:45:55,005 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517954832_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:55,005 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_STARTED]: dagID=dag_1463517954832_0001_1, startTime=1463517955005
2016-05-17 20:45:55,005 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:55,005 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:55,006 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517954832_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:55,007 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E80C71241AB144E3807B712750E7D22B : {6DBAD42B0AD64641B5A2DC3C90FB0E50={InputName=6DBAD42B0AD64641B5A2DC3C90FB0E50}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:55,007 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6DBAD42B0AD64641B5A2DC3C90FB0E50, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:55,008 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,008 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463517954832_0001, dagName=[7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace
2016-05-17 20:45:55,008 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,008 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,008 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]. Starting root input initializers: 1
2016-05-17 20:45:55,011 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:55,011 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6DBAD42B0AD64641B5A2DC3C90FB0E50 on vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,013 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:55,013 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6DBAD42B0AD64641B5A2DC3C90FB0E50 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:55,013 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:55,014 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:55,015 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:55,016 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6DBAD42B0AD64641B5A2DC3C90FB0E50 on vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,016 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:55,016 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6DBAD42B0AD64641B5A2DC3C90FB0E50=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:55,016 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] parallelism set to 1
2016-05-17 20:45:55,017 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:55,017 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517954832_0001_1_00, eventCount=1
2016-05-17 20:45:55,017 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,017 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4AE0D34413814F0E9E38CAB108E48018, vertexId=vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:55,017 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4AE0D34413814F0E9E38CAB108E48018, vertexId=vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,019 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4AE0D34413814F0E9E38CAB108E48018, vertexName=E80C71241AB144E3807B712750E7D22B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:55,019 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4AE0D34413814F0E9E38CAB108E48018, vertexId=vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,021 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E80C71241AB144E3807B712750E7D22B, vertexId=vertex_1463517954832_0001_1_00, initRequestedTime=1463517955007, initedTime=1463517955021, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:55,021 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] with distanceFromRoot: 0
2016-05-17 20:45:55,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:55,022 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:55,022 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E80C71241AB144E3807B712750E7D22B
2016-05-17 20:45:55,023 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,023 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:55,024 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517954832_0001_1_00, startRequestedTime=1463517955022, startedTime=1463517955022
2016-05-17 20:45:55,024 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:55,025 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:55,025 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:TASK_STARTED]: vertexName=E80C71241AB144E3807B712750E7D22B, taskId=task_1463517954832_0001_1_00_000000, scheduledTime=1463517955025, launchTime=1463517955025
2016-05-17 20:45:55,026 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517954832_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:55,026 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,026 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517954832_0001, with dag id: dag_1463517954832_0001_1
2016-05-17 20:45:55,028 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace, VertexName: E80C71241AB144E3807B712750E7D22B, VertexParallelism: 1, TaskAttemptID:attempt_1463517954832_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6DBAD42B0AD64641B5A2DC3C90FB0E50, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=4AE0D34413814F0E9E38CAB108E48018, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:55,029 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517954832_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:55,030 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:55,033 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:55,033 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:55,033 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:55,034 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517954832_0001_00_000001
2016-05-17 20:45:55,034 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:55,036 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517954832_0001_00_000001
2016-05-17 20:45:55,036 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:55,039 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:55,039 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:55,039 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517954832_0001_00_000001, launchTime=1463517955039
2016-05-17 20:45:55,041 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517954832_0001_00_000001
2016-05-17 20:45:55,042 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517954832_0001_1_00_000000_0] to container: [container_1463517954832_0001_00_000001]
2016-05-17 20:45:55,042 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:55,042 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517954832_0001_00_000001 given task: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,043 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517954832_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,043 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517954832_0001_1_00_000000_0] started. Is using containerId: [container_1463517954832_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:55,043 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E80C71241AB144E3807B712750E7D22B, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0, startTime=1463517955042, containerId=container_1463517954832_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517954832_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:55,043 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517954832_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:55,044 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517954832_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:55,044 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:55 Starting to run new task attempt: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,045 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,045 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,046 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:55,046 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:55,046 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:55,047 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:55,047 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:55,048 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace, VertexName: E80C71241AB144E3807B712750E7D22B, VertexParallelism: 1, TaskAttemptID:attempt_1463517954832_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6DBAD42B0AD64641B5A2DC3C90FB0E50, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=4AE0D34413814F0E9E38CAB108E48018, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:55,048 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:55,049 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,049 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:55,049 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,049 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6DBAD42B0AD64641B5A2DC3C90FB0E50, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:55,050 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:55,050 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,050 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,052 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,053 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4AE0D34413814F0E9E38CAB108E48018, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:55,053 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:55,053 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4AE0D34413814F0E9E38CAB108E48018
2016-05-17 20:45:55,053 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:55,054 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4AE0D34413814F0E9E38CAB108E48018
2016-05-17 20:45:55,052 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517954832_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:55,056 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:55,055 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:55,059 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,060 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,066 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E80C71241AB144E3807B712750E7D22B, ordinal: 0
2016-05-17 20:45:55,066 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E80C71241AB144E3807B712750E7D22B, mem on start (mb), free: 348, total: 652, max: 672
2016-05-17 20:45:55,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,067 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:55,067 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:55,069 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:55,069 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4AE0D34413814F0E9E38CAB108E48018
2016-05-17 20:45:55,069 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:55,070 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:55,070 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:55,070 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:55,071 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:55,071 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:55,071 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:55,072 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:55,072 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6DBAD42B0AD64641B5A2DC3C90FB0E50, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:55,072 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4AE0D34413814F0E9E38CAB108E48018, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:55,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:55,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,073 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:55,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6DBAD42B0AD64641B5A2DC3C90FB0E50 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:55,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:55,075 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:55,075 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:55,076 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,076 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:55,077 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1ee9e51b
2016-05-17 20:45:55,078 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1ee9e51b
2016-05-17 20:45:55,078 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:55,078 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:55,079 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:55,080 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,080 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"], id: 4AE0D34413814F0E9E38CAB108E48018
2016-05-17 20:45:55,081 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-17 20:45:55,081 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:55,081 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:55,082 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E80C71241AB144E3807B712750E7D22B, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:55,083 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:55,084 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:55,084 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,084 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517954832_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:55,085 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,085 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E80C71241AB144E3807B712750E7D22B, mem on close (mb), free: 346, total: 652, max: 672
2016-05-17 20:45:55 Completed running task attempt: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,086 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:55,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-17 20:45:55,089 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:55,089 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,089 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:55,089 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E80C71241AB144E3807B712750E7D22B, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0, startTime=1463517955042, finishTime=1463517955089, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:55,089 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517954832_0001_00_000001
2016-05-17 20:45:55,090 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517954832_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:55,090 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:55,090 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,091 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:55,091 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:TASK_FINISHED]: vertexName=E80C71241AB144E3807B712750E7D22B, taskId=task_1463517954832_0001_1_00_000000, startTime=1463517955042, finishTime=1463517955091, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463517954832_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:55,091 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517954832_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:55,092 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] : 1
2016-05-17 20:45:55,092 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,093 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:55,093 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:VERTEX_FINISHED]: vertexName=E80C71241AB144E3807B712750E7D22B, vertexId=vertex_1463517954832_0001_1_00, initRequestedTime=1463517955007, initedTime=1463517955021, startRequestedTime=1463517955022, startedTime=1463517955022, finishTime=1463517955092, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463517955042, firstTasksToStart=[ task_1463517954832_0001_1_00_000000 ], lastTaskFinishTime=1463517955089, lastTasksToFinish=[ task_1463517954832_0001_1_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517954832_0001_1_00_000000 ], longestDurationTasks=[ task_1463517954832_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:55,093 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:55,094 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:55,094 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517954832_0001_00_000001, stoppedTime=1463517955094, exitStatus=0
2016-05-17 20:45:55,094 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:55,094 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:55,094 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:55,094 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517954832_0001_1
2016-05-17 20:45:55,094 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517954832_0001_1
2016-05-17 20:45:55,095 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 4AE0D34413814F0E9E38CAB108E48018 for vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,101 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517954832_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:55,101 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517954832_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:55,101 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517954832_0001_00_000001, Exiting
2016-05-17 20:45:55,101 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517954832_0001_00_000001
2016-05-17 20:45:55,101 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517954832_0001_00_000001
2016-05-17 20:45:55,103 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517954832_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:55,103 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517954832_0001_00_000001 completed successfully
2016-05-17 20:45:55,105 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517954832_0001_1, queueSize=0
2016-05-17 20:45:55,105 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:55,106 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517954832_0001_1, startTime=1463517955005, finishTime=1463517955103, timeTaken=98, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:55,106 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517954832_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:55,106 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517954832_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:55,107 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517954832_0001_00_000001
2016-05-17 20:45:55,107 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:55 Completed Dag: dag_1463517954832_0001_1
2016-05-17 20:45:55,109 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517954832_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:55,110 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517954832_0001_1_post
2016-05-17 20:45:55,110 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,111 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:55,113 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463517954832_0001
2016-05-17 20:45:55,115 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:55,115 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:55,115 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:55,115 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:55,115 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:55,116 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:55,116 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:55,116 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:55,117 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:55,117 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:55,117 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:55,118 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:55,118 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:55,122 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001
2016-05-17 20:45:55,122 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace/_temporary
2016-05-17 20:45:55,123 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.310
2016-05-17 20:45:55,155 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-17 20:45:55,188 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:55,189 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:55,224 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 20:45:55,236 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-17 20:45:55,237 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:55,237 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:55,237 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:55,255 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 20:45:55,261 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-17 20:45:55,291 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:55,292 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:55,306 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 20:45:55,307 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:55,307 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-17 20:45:55,308 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  parallel execution of steps is enabled: false
2016-05-17 20:45:55,308 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  executing total steps: 1
2016-05-17 20:45:55,308 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  allocating management threads: 1
2016-05-17 20:45:55,310 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount2/count2
2016-05-17 20:45:55,311 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:55,322 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 20:45:55,327 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:55,327 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:55,328 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:55,338 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001 doesn't exist and is created
2016-05-17 20:45:55,356 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:55,356 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:55,357 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001
2016-05-17 20:45:55,360 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517955327_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:55,370 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:55,371 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:55,372 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:55,373 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:55,373 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001/localmode-log-dir
2016-05-17 20:45:55,373 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001/localmode-log-dir/history.txt.appattempt_1463517955327_0001_000000, maxErrors=10
2016-05-17 20:45:55,373 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:55,374 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517955327_0001_000000, appSubmitTime=1463517955359, launchTime=1463517955359
2016-05-17 20:45:55,376 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:55,377 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:55,379 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:55,379 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:55,385 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35668
2016-05-17 20:45:55,387 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:55,389 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517955327_0001_000000, startTime=1463517955389
2016-05-17 20:45:55,389 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:55,390 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:55,456 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:55,457 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:55,457 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463517955327_0001, dagName=[C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2
2016-05-17 20:45:55,475 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2
2016-05-17 20:45:55,477 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517955327_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001/localmode-log-dir/dag_1463517955327_0001_1.dot
2016-05-17 20:45:55,477 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001/localmode-log-dir/dag_1463517955327_0001_1-tez-dag.pb.txt
2016-05-17 20:45:55,485 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517955327_0001_1
2016-05-17 20:45:55,485 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,485 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2
2016-05-17 20:45:55 Running Dag: dag_1463517955327_0001_1
2016-05-17 20:45:55,496 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517955327_0001_1, submitTime=1463517955475
2016-05-17 20:45:55,496 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:55,497 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,497 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:55,497 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517955327_0001_1, initTime=1463517955496
2016-05-17 20:45:55,497 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955327_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:55,497 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:55,498 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_STARTED]: dagID=dag_1463517955327_0001_1, startTime=1463517955498
2016-05-17 20:45:55,498 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:55,498 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:55,498 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955327_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:55,499 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D4C620047B804607BE524C3D6E8A1D1E : {8959902C54374501A5283B13E8C43754={InputName=8959902C54374501A5283B13E8C43754}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:55,499 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8959902C54374501A5283B13E8C43754, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:55,499 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,499 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463517955327_0001, dagName=[C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2
2016-05-17 20:45:55,499 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,499 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,500 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]. Starting root input initializers: 1
2016-05-17 20:45:55,500 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:55,500 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,501 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8959902C54374501A5283B13E8C43754 on vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,503 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:55,503 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:55,504 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,504 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,504 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,505 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7448873726D14BBB9B8229B6DCBDB440, vertexId=vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:55,505 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7448873726D14BBB9B8229B6DCBDB440, vertexId=vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,504 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8959902C54374501A5283B13E8C43754 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:55,506 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:55,506 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:55,507 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7448873726D14BBB9B8229B6DCBDB440, vertexName=DD516F6A13164CD2BF1515BC64D72E2A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:55,508 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7448873726D14BBB9B8229B6DCBDB440, vertexId=vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,508 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 20:45:55,509 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8959902C54374501A5283B13E8C43754 on vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,509 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:55,510 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, vertexId=vertex_1463517955327_0001_1_01, initRequestedTime=1463517955500, initedTime=1463517955509, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:55,510 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:55,510 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:55,510 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8959902C54374501A5283B13E8C43754=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:55,511 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] parallelism set to 1
2016-05-17 20:45:55,511 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:55,511 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517955327_0001_1_00, eventCount=1
2016-05-17 20:45:55,512 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:55,512 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, vertexId=vertex_1463517955327_0001_1_00, initRequestedTime=1463517955499, initedTime=1463517955512, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:55,512 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] with distanceFromRoot: 0
2016-05-17 20:45:55,513 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:55,513 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,513 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,514 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:55,514 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517955327_0001_1_00, startRequestedTime=1463517955513, startedTime=1463517955513
2016-05-17 20:45:55,515 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:55,515 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:55,516 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_STARTED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, taskId=task_1463517955327_0001_1_00_000000, scheduledTime=1463517955515, launchTime=1463517955515
2016-05-17 20:45:55,516 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:55,516 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517955327_0001_1_00 for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] numStartedSources: 1 numSources: 1
2016-05-17 20:45:55,516 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] with distanceFromRoot: 1
2016-05-17 20:45:55,517 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,517 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D4C620047B804607BE524C3D6E8A1D1E in vertex: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,517 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DD516F6A13164CD2BF1515BC64D72E2A with 1 source tasks and 1 pending tasks
2016-05-17 20:45:55,518 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:55,518 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517955327_0001_1_01, startRequestedTime=1463517955517, startedTime=1463517955517
2016-05-17 20:45:55,518 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:55,519 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2, VertexName: D4C620047B804607BE524C3D6E8A1D1E, VertexParallelism: 1, TaskAttemptID:attempt_1463517955327_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8959902C54374501A5283B13E8C43754, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD516F6A13164CD2BF1515BC64D72E2A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:55,519 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:55,519 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463517955327_0001, with dag id: dag_1463517955327_0001_1
2016-05-17 20:45:55,520 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:55,523 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:55,524 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:55,524 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:55,525 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517955327_0001_00_000001
2016-05-17 20:45:55,525 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:55,528 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517955327_0001_00_000001
2016-05-17 20:45:55,528 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:55,529 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:55,529 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:55,529 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517955327_0001_00_000001, launchTime=1463517955529
2016-05-17 20:45:55,530 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955327_0001_00_000001
2016-05-17 20:45:55,532 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517955327_0001_1_00_000000_0] to container: [container_1463517955327_0001_00_000001]
2016-05-17 20:45:55,532 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:55,532 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517955327_0001_00_000001 given task: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,532 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955327_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,532 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517955327_0001_1_00_000000_0] started. Is using containerId: [container_1463517955327_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:55,533 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0, startTime=1463517955532, containerId=container_1463517955327_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517955327_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:55,533 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:55,533 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:55,533 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:55 Starting to run new task attempt: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,534 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,534 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,534 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:55,534 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:55,535 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:55,535 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:55,536 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:55,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2, VertexName: D4C620047B804607BE524C3D6E8A1D1E, VertexParallelism: 1, TaskAttemptID:attempt_1463517955327_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8959902C54374501A5283B13E8C43754, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD516F6A13164CD2BF1515BC64D72E2A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:55,536 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:55,536 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,537 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:55,537 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,537 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517955327_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:55,538 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,538 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8959902C54374501A5283B13E8C43754, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:55,540 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:55,543 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DD516F6A13164CD2BF1515BC64D72E2A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:55,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:55,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,546 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:55,546 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,546 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,547 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:55,547 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,552 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,552 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D4C620047B804607BE524C3D6E8A1D1E, ordinal: 0
2016-05-17 20:45:55,552 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4C620047B804607BE524C3D6E8A1D1E, mem on start (mb), free: 266, total: 652, max: 672
2016-05-17 20:45:55,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:55,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:55,553 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:55,553 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:55,554 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:55,554 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:55,555 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:55,555 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:55,555 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:55,555 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:55,556 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8959902C54374501A5283B13E8C43754, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:55,556 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DD516F6A13164CD2BF1515BC64D72E2A, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:55,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:55,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8959902C54374501A5283B13E8C43754 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:55,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,557 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:55,557 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,557 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:55,558 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:55,558 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:55,559 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,562 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:55,562 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,562 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: E801256111E8442FA8FA072793C4C3B5
2016-05-17 20:45:55,563 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] E801256111E8442FA8FA072793C4C3B5
2016-05-17 20:45:55,565 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2111bc5f; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@136611ef; comparator=cascading.tuple.hadoop.util.TupleComparator@12236bdc; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:55,565 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:55,566 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:55,566 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:55,579 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:55,579 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@37dd7c4e
2016-05-17 20:45:55,580 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@37dd7c4e
2016-05-17 20:45:55,581 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:55,581 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:55,591 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:55,592 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:55,592 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:55,592 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:55,592 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:55,594 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:55,594 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:55,595 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D4C620047B804607BE524C3D6E8A1D1E, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:55,602 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,602 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4C620047B804607BE524C3D6E8A1D1E, mem on close (mb), free: 323, total: 653, max: 672
2016-05-17 20:45:55,602 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:55,603 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:55,603 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 20:45:55,603 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 20:45:55,607 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:55 Completed running task attempt: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,611 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:55,612 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517955327_0001_1_00, eventCount=1
2016-05-17 20:45:55,612 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:55,612 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0, startTime=1463517955532, finishTime=1463517955612, timeTaken=80, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955595, Process_Duration=7, Process_End_Time=1463517955602, Read_Duration=5, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:55,612 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:55,613 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,613 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:55,613 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_FINISHED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, taskId=task_1463517955327_0001_1_00_000000, startTime=1463517955532, finishTime=1463517955613, timeTaken=81, status=SUCCEEDED, successfulAttemptID=attempt_1463517955327_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955595, Process_Duration=7, Process_End_Time=1463517955602, Read_Duration=5, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:55,613 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:55,614 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:55,614 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] : 1
2016-05-17 20:45:55,614 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,615 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:55,615 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_FINISHED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, vertexId=vertex_1463517955327_0001_1_00, initRequestedTime=1463517955499, initedTime=1463517955512, startRequestedTime=1463517955513, startedTime=1463517955513, finishTime=1463517955614, timeTaken=101, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955595, Process_Duration=7, Process_End_Time=1463517955602, Read_Duration=5, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463517955532, firstTasksToStart=[ task_1463517955327_0001_1_00_000000 ], lastTaskFinishTime=1463517955612, lastTasksToFinish=[ task_1463517955327_0001_1_00_000000 ], minTaskDuration=80, maxTaskDuration=80, avgTaskDuration=80.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517955327_0001_1_00_000000 ], longestDurationTasks=[ task_1463517955327_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:55,616 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:55,616 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:55,616 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517955327_0001_00_000001, stoppedTime=1463517955616, exitStatus=0
2016-05-17 20:45:55,616 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:55,616 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:55,616 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] attempt: attempt_1463517955327_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:55,617 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,617 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,617 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:55,618 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517955327_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:55,618 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_STARTED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, taskId=task_1463517955327_0001_1_01_000000, scheduledTime=1463517955618, launchTime=1463517955618
2016-05-17 20:45:55,619 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:55,619 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:55,619 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,620 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2, VertexName: DD516F6A13164CD2BF1515BC64D72E2A, VertexParallelism: 1, TaskAttemptID:attempt_1463517955327_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4C620047B804607BE524C3D6E8A1D1E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7448873726D14BBB9B8229B6DCBDB440, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:55,618 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:55,621 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:55,621 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:55,622 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517955327_0001_00_000002
2016-05-17 20:45:55,622 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:55,624 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517955327_0001_00_000002
2016-05-17 20:45:55,624 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:55,625 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:55,625 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517955327_0001_00_000002, launchTime=1463517955624
2016-05-17 20:45:55,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517955595, Process_Duration=7, Process_End_Time=1463517955602, Read_Duration=5, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 20:45:55,625 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:55,626 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955327_0001_00_000002
2016-05-17 20:45:55,626 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:55,626 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,626 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517955327_0001_1_01_000000_0] to container: [container_1463517955327_0001_00_000002]
2016-05-17 20:45:55,627 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:55,627 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517955327_0001_1_01_000000_0] started. Is using containerId: [container_1463517955327_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:55,627 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0, startTime=1463517955627, containerId=container_1463517955327_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517955327_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:55,627 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:55,628 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:55,626 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955327_0001_00_000001
2016-05-17 20:45:55,628 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517955327_0001_00_000002 given task: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,628 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517955327_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:55,628 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:55,628 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955327_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:55,628 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955327_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,628 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517955327_0001_00_000001, Exiting
2016-05-17 20:45:55,629 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517955327_0001_00_000001
2016-05-17 20:45:55,629 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517955327_0001_00_000001
2016-05-17 20:45:55 Starting to run new task attempt: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,630 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,630 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,630 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:55,630 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:55,630 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:55,631 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:55,631 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:55,631 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2, VertexName: DD516F6A13164CD2BF1515BC64D72E2A, VertexParallelism: 1, TaskAttemptID:attempt_1463517955327_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4C620047B804607BE524C3D6E8A1D1E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7448873726D14BBB9B8229B6DCBDB440, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:55,631 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:55,631 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517955327_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:55,631 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517955327_0001_00_000001 completed successfully
2016-05-17 20:45:55,632 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:55,632 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,634 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:55,634 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,634 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517955327_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:55,636 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,638 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:55,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7448873726D14BBB9B8229B6DCBDB440, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:55,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D4C620047B804607BE524C3D6E8A1D1E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:55,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:55,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:55,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7448873726D14BBB9B8229B6DCBDB440
2016-05-17 20:45:55,643 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:55,644 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7448873726D14BBB9B8229B6DCBDB440
2016-05-17 20:45:55,644 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:55,645 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:55,645 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,644 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD516F6A13164CD2BF1515BC64D72E2A, ordinal: 1
2016-05-17 20:45:55,646 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD516F6A13164CD2BF1515BC64D72E2A, mem on start (mb), free: 318, total: 653, max: 672
2016-05-17 20:45:55,646 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,648 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:55,646 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,648 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:55,654 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:55,654 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7448873726D14BBB9B8229B6DCBDB440
2016-05-17 20:45:55,655 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:55,655 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:55,656 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:55,656 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:55,656 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:55,657 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:55,658 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:55,658 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7448873726D14BBB9B8229B6DCBDB440, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:55,658 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D4C620047B804607BE524C3D6E8A1D1E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:55,658 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:55,658 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D4C620047B804607BE524C3D6E8A1D1E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:55,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:55,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:55,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,660 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:55,661 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:55,662 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D4C620047B804607BE524C3D6E8A1D1E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:55,662 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:55,662 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:55,662 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:55,663 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,663 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:55,663 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,667 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517955327_0001_1_00_000000_0_10067, runDuration: 64495, ]
2016-05-17 20:45:55,667 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 20:45:55,668 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: E801256111E8442FA8FA072793C4C3B5
2016-05-17 20:45:55,668 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"], id: 7448873726D14BBB9B8229B6DCBDB440
2016-05-17 20:45:55,669 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-17 20:45:55,669 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] E801256111E8442FA8FA072793C4C3B5, for 1 inputs
2016-05-17 20:45:55,669 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517955327_0001_1_00_000000_0_10067] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463517955669, TimeTaken=1, Rate=0.44 MB/s
2016-05-17 20:45:55,669 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,670 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:55,670 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D4C620047B804607BE524C3D6E8A1D1E] #11 in 2ms
2016-05-17 20:45:55,670 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D4C620047B804607BE524C3D6E8A1D1E] #11, status:false, isInterrupted:false
2016-05-17 20:45:55,671 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:55,671 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 20:45:55,671 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:55,672 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:55,672 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 20:45:55,672 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,672 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:55,673 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD516F6A13164CD2BF1515BC64D72E2A, all 1 inputs ready in: 00:00:00.004
2016-05-17 20:45:55,677 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@317db632; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@3444a600
2016-05-17 20:45:55,680 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:55,680 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:55,680 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,681 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517955327_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:55,681 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,681 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD516F6A13164CD2BF1515BC64D72E2A, mem on close (mb), free: 315, total: 653, max: 672
2016-05-17 20:45:55,682 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,682 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:55,683 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:55,683 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:55,683 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:55,683 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:55 Completed running task attempt: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,684 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:55,685 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:55,685 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0, startTime=1463517955627, finishTime=1463517955685, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955674, Process_Duration=7, Process_End_Time=1463517955681, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:55,686 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:55,686 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,686 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:55,686 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:55,687 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_FINISHED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, taskId=task_1463517955327_0001_1_01_000000, startTime=1463517955627, finishTime=1463517955686, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1463517955327_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955674, Process_Duration=7, Process_End_Time=1463517955681, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:55,687 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:55,687 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] : 1
2016-05-17 20:45:55,687 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,688 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:55,688 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_FINISHED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, vertexId=vertex_1463517955327_0001_1_01, initRequestedTime=1463517955500, initedTime=1463517955509, startRequestedTime=1463517955517, startedTime=1463517955517, finishTime=1463517955687, timeTaken=170, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955674, Process_Duration=7, Process_End_Time=1463517955681, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463517955627, firstTasksToStart=[ task_1463517955327_0001_1_01_000000 ], lastTaskFinishTime=1463517955685, lastTasksToFinish=[ task_1463517955327_0001_1_01_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517955327_0001_1_01_000000 ], longestDurationTasks=[ task_1463517955327_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:55,689 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:55,689 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:55,689 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517955327_0001_00_000002, stoppedTime=1463517955689, exitStatus=0
2016-05-17 20:45:55,689 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:55,690 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:55,690 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:55,690 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517955327_0001_1
2016-05-17 20:45:55,690 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517955327_0001_1
2016-05-17 20:45:55,691 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 7448873726D14BBB9B8229B6DCBDB440 for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,690 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517955674, Process_Duration=7, Process_End_Time=1463517955681, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 20:45:55,692 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:55,692 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,692 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955327_0001_00_000002
2016-05-17 20:45:55,693 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517955327_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:55,693 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955327_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:55,693 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517955327_0001_00_000002, Exiting
2016-05-17 20:45:55,693 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517955327_0001_00_000002
2016-05-17 20:45:55,693 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517955327_0001_00_000002
2016-05-17 20:45:55,694 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517955327_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:55,694 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517955327_0001_00_000002 completed successfully
2016-05-17 20:45:55,699 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,702 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517955327_0001_1, queueSize=0
2016-05-17 20:45:55,702 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:55,704 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517955327_0001_1, startTime=1463517955498, finishTime=1463517955699, timeTaken=201, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1369440256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035911269, Process_Duration=14, Process_End_Time=2927035911283, Read_Duration=5, Tuples_Read=40, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 20:45:55,704 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517955327_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:55,705 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955327_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:55,707 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517955327_0001_00_000002
2016-05-17 20:45:55,708 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:55 Completed Dag: dag_1463517955327_0001_1
2016-05-17 20:45:55,710 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517955327_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:55,711 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517955327_0001_1_post
2016-05-17 20:45:55,711 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,712 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:55,714 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463517955327_0001
2016-05-17 20:45:55,718 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:55,718 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:55,718 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:55,718 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:55,718 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:55,719 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:55,719 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:55,720 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:55,720 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:55,720 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:55,720 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:55,720 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:55,719 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:55,726 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001
2016-05-17 20:45:55,726 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2/_temporary
2016-05-17 20:45:55,727 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  completed in: 00:00.420
2016-05-17 20:45:55,799 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-17 20:45:55,829 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:55,829 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:55,863 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 20:45:55,882 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-17 20:45:55,884 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:55,884 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:55,884 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:55,905 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 20:45:55,911 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-17 20:45:55,955 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 20:45:55,955 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:55,955 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-17 20:45:55,956 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  parallel execution of steps is enabled: false
2016-05-17 20:45:55,956 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  executing total steps: 1
2016-05-17 20:45:55,956 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  allocating management threads: 1
2016-05-17 20:45:55,957 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount3/count3
2016-05-17 20:45:55,957 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:55,968 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 20:45:55,973 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:55,973 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:55,973 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:55,982 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001 doesn't exist and is created
2016-05-17 20:45:55,997 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:55,998 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:55,998 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001
2016-05-17 20:45:56,001 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517955973_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:56,011 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:56,012 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:56,013 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:56,014 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:56,014 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001/localmode-log-dir
2016-05-17 20:45:56,014 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001/localmode-log-dir/history.txt.appattempt_1463517955973_0001_000000, maxErrors=10
2016-05-17 20:45:56,015 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:56,015 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517955973_0001_000000, appSubmitTime=1463517956001, launchTime=1463517956001
2016-05-17 20:45:56,019 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:56,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:56,022 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:56,022 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:56,022 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50231
2016-05-17 20:45:56,026 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:56,029 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517955973_0001_000000, startTime=1463517956029
2016-05-17 20:45:56,029 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:56,029 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:56,098 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:56,098 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:56,098 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463517955973_0001, dagName=[2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3
2016-05-17 20:45:56,114 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3
2016-05-17 20:45:56,116 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517955973_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001/localmode-log-dir/dag_1463517955973_0001_1.dot
2016-05-17 20:45:56,117 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001/localmode-log-dir/dag_1463517955973_0001_1-tez-dag.pb.txt
2016-05-17 20:45:56,128 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517955973_0001_1
2016-05-17 20:45:56,129 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,129 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3
2016-05-17 20:45:56 Running Dag: dag_1463517955973_0001_1
2016-05-17 20:45:56,143 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517955973_0001_1, submitTime=1463517956115
2016-05-17 20:45:56,143 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:56,144 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,144 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:56,144 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517955973_0001_1, initTime=1463517956143
2016-05-17 20:45:56,144 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:56,145 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955973_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:56,146 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_STARTED]: dagID=dag_1463517955973_0001_1, startTime=1463517956146
2016-05-17 20:45:56,146 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:56,146 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:56,146 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955973_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:56,146 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E97316F327FA4D2D84F09F72A535DE7D : {082D7BE07AB1443F9FA3B873961105C2={InputName=082D7BE07AB1443F9FA3B873961105C2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:56,147 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 082D7BE07AB1443F9FA3B873961105C2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:56,147 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,147 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,147 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,148 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463517955973_0001, dagName=[2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3
2016-05-17 20:45:56,148 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]. Starting root input initializers: 1
2016-05-17 20:45:56,148 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:56,148 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,150 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 082D7BE07AB1443F9FA3B873961105C2 on vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,151 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:56,151 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,152 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,152 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,152 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=DA0740327CBD4EC1994A080E5DD4B3A3, vertexId=vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:56,152 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:56,153 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=DA0740327CBD4EC1994A080E5DD4B3A3, vertexId=vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,153 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 082D7BE07AB1443F9FA3B873961105C2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:56,154 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:56,154 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:56,155 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=DA0740327CBD4EC1994A080E5DD4B3A3, vertexName=A3F494A478E84615BDF9C31E63BA6EDC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:56,155 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=DA0740327CBD4EC1994A080E5DD4B3A3, vertexId=vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,157 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 20:45:56,157 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:56,157 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, vertexId=vertex_1463517955973_0001_1_01, initRequestedTime=1463517956148, initedTime=1463517956157, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:56,157 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:56,158 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 082D7BE07AB1443F9FA3B873961105C2 on vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,158 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:56,159 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {082D7BE07AB1443F9FA3B873961105C2=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:56,159 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] parallelism set to 1
2016-05-17 20:45:56,160 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:56,160 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517955973_0001_1_00, eventCount=1
2016-05-17 20:45:56,161 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, vertexId=vertex_1463517955973_0001_1_00, initRequestedTime=1463517956146, initedTime=1463517956161, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:56,161 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] with distanceFromRoot: 0
2016-05-17 20:45:56,161 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:56,162 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:56,162 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,162 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,162 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:56,163 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517955973_0001_1_00, startRequestedTime=1463517956162, startedTime=1463517956162
2016-05-17 20:45:56,163 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:56,163 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:56,163 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_STARTED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, taskId=task_1463517955973_0001_1_00_000000, scheduledTime=1463517956163, launchTime=1463517956163
2016-05-17 20:45:56,164 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:56,164 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517955973_0001_1_00 for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] numStartedSources: 1 numSources: 1
2016-05-17 20:45:56,164 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] with distanceFromRoot: 1
2016-05-17 20:45:56,164 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,166 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E97316F327FA4D2D84F09F72A535DE7D in vertex: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,166 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A3F494A478E84615BDF9C31E63BA6EDC with 1 source tasks and 1 pending tasks
2016-05-17 20:45:56,166 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463517955973_0001, with dag id: dag_1463517955973_0001_1
2016-05-17 20:45:56,168 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:56,168 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517955973_0001_1_01, startRequestedTime=1463517956166, startedTime=1463517956166
2016-05-17 20:45:56,169 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:56,169 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3, VertexName: E97316F327FA4D2D84F09F72A535DE7D, VertexParallelism: 1, TaskAttemptID:attempt_1463517955973_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=082D7BE07AB1443F9FA3B873961105C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A3F494A478E84615BDF9C31E63BA6EDC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:56,170 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:56,172 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:56,173 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:56,173 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:56,173 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:56,174 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517955973_0001_00_000001
2016-05-17 20:45:56,174 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:56,176 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517955973_0001_00_000001
2016-05-17 20:45:56,176 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:56,177 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:56,177 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:56,177 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517955973_0001_00_000001, launchTime=1463517956177
2016-05-17 20:45:56,178 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955973_0001_00_000001
2016-05-17 20:45:56,178 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517955973_0001_1_00_000000_0] to container: [container_1463517955973_0001_00_000001]
2016-05-17 20:45:56,178 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:56,179 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517955973_0001_00_000001 given task: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,179 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955973_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,179 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517955973_0001_1_00_000000_0] started. Is using containerId: [container_1463517955973_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:56,179 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0, startTime=1463517956179, containerId=container_1463517955973_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517955973_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:56,179 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:56,179 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:56,179 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,179 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,180 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,180 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:56,180 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:56,180 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:56,180 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:56,180 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:56,180 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3, VertexName: E97316F327FA4D2D84F09F72A535DE7D, VertexParallelism: 1, TaskAttemptID:attempt_1463517955973_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=082D7BE07AB1443F9FA3B873961105C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A3F494A478E84615BDF9C31E63BA6EDC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:56,181 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:56,181 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,181 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:56,181 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,182 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=082D7BE07AB1443F9FA3B873961105C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:56,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:56,183 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,183 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,185 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,189 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517955973_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:56,190 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A3F494A478E84615BDF9C31E63BA6EDC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:56,190 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:56,190 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,192 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:56,192 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:56,194 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,195 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,195 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:56,195 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:56,196 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:56,196 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,197 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,199 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E97316F327FA4D2D84F09F72A535DE7D, ordinal: 0
2016-05-17 20:45:56,199 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E97316F327FA4D2D84F09F72A535DE7D, mem on start (mb), free: 226, total: 653, max: 672
2016-05-17 20:45:56,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:56,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:56,199 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:56,199 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:56,200 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:56,200 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:56,200 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:56,200 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:56,200 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:56,201 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 082D7BE07AB1443F9FA3B873961105C2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:56,201 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A3F494A478E84615BDF9C31E63BA6EDC, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:56,201 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:56,201 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 082D7BE07AB1443F9FA3B873961105C2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:56,202 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:56,202 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:56,202 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,202 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:56,203 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,203 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:56,204 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,205 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:56,205 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6f40f6c5
2016-05-17 20:45:56,207 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6f40f6c5
2016-05-17 20:45:56,208 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:56,208 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:56,207 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,209 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,210 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 31FFDA18ACE24D1D8BDD6EAFAA2226C0
2016-05-17 20:45:56,210 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 31FFDA18ACE24D1D8BDD6EAFAA2226C0
2016-05-17 20:45:56,212 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@b9641a6; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@35afff3b; comparator=cascading.tuple.hadoop.util.TupleComparator@359d9606; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:56,213 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:56,213 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:56,214 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:56,238 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:56,239 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:56,239 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:56,239 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:56,240 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:56,241 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,241 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:56,241 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E97316F327FA4D2D84F09F72A535DE7D, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:56,244 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,245 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E97316F327FA4D2D84F09F72A535DE7D, mem on close (mb), free: 323, total: 653, max: 672
2016-05-17 20:45:56,245 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:56,245 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:56,245 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 20:45:56,246 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 20:45:56,250 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,254 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:56,256 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517955973_0001_1_00, eventCount=1
2016-05-17 20:45:56,256 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:56,257 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0, startTime=1463517956179, finishTime=1463517956256, timeTaken=77, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956242, Process_Duration=2, Process_End_Time=1463517956244, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:56,257 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:56,257 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,257 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:56,257 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:56,257 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_FINISHED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, taskId=task_1463517955973_0001_1_00_000000, startTime=1463517956179, finishTime=1463517956257, timeTaken=78, status=SUCCEEDED, successfulAttemptID=attempt_1463517955973_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956242, Process_Duration=2, Process_End_Time=1463517956244, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:56,258 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:56,258 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] : 1
2016-05-17 20:45:56,258 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,259 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:56,259 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_FINISHED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, vertexId=vertex_1463517955973_0001_1_00, initRequestedTime=1463517956146, initedTime=1463517956161, startRequestedTime=1463517956162, startedTime=1463517956162, finishTime=1463517956258, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956242, Process_Duration=2, Process_End_Time=1463517956244, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463517956179, firstTasksToStart=[ task_1463517955973_0001_1_00_000000 ], lastTaskFinishTime=1463517956256, lastTasksToFinish=[ task_1463517955973_0001_1_00_000000 ], minTaskDuration=77, maxTaskDuration=77, avgTaskDuration=77.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517955973_0001_1_00_000000 ], longestDurationTasks=[ task_1463517955973_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:56,259 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:56,260 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:56,260 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] attempt: attempt_1463517955973_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:56,260 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,260 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,260 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517955973_0001_00_000001, stoppedTime=1463517956260, exitStatus=0
2016-05-17 20:45:56,261 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:56,261 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:56,261 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:56,261 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_STARTED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, taskId=task_1463517955973_0001_1_01_000000, scheduledTime=1463517956261, launchTime=1463517956261
2016-05-17 20:45:56,261 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:56,261 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517956242, Process_Duration=2, Process_End_Time=1463517956244, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 20:45:56,261 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:56,262 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517955973_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:56,262 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:56,262 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,262 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,263 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:56,263 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3, VertexName: A3F494A478E84615BDF9C31E63BA6EDC, VertexParallelism: 1, TaskAttemptID:attempt_1463517955973_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E97316F327FA4D2D84F09F72A535DE7D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA0740327CBD4EC1994A080E5DD4B3A3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:56,263 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:56,263 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:56,265 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517955973_0001_00_000002
2016-05-17 20:45:56,265 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:56,266 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955973_0001_00_000001
2016-05-17 20:45:56,266 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517955973_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:56,266 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955973_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:56,266 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517955973_0001_00_000001, Exiting
2016-05-17 20:45:56,266 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517955973_0001_00_000001
2016-05-17 20:45:56,266 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517955973_0001_00_000001
2016-05-17 20:45:56,268 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517955973_0001_00_000002
2016-05-17 20:45:56,268 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517955973_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:56,268 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517955973_0001_00_000001 completed successfully
2016-05-17 20:45:56,268 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:56,268 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:56,270 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:56,270 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:56,270 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517955973_0001_00_000002, launchTime=1463517956270
2016-05-17 20:45:56,271 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955973_0001_00_000002
2016-05-17 20:45:56,272 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517955973_0001_1_01_000000_0] to container: [container_1463517955973_0001_00_000002]
2016-05-17 20:45:56,272 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:56,272 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517955973_0001_1_01_000000_0] started. Is using containerId: [container_1463517955973_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:56,272 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0, startTime=1463517956272, containerId=container_1463517955973_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517955973_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:56,273 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:56,273 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:56,273 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:56,273 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517955973_0001_00_000002 given task: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,273 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955973_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,273 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,274 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,274 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:56,274 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:56,274 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:56,274 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:56,274 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:56,274 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3, VertexName: A3F494A478E84615BDF9C31E63BA6EDC, VertexParallelism: 1, TaskAttemptID:attempt_1463517955973_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E97316F327FA4D2D84F09F72A535DE7D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA0740327CBD4EC1994A080E5DD4B3A3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:56,275 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:56,275 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,275 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:56,275 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,276 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517955973_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:56,276 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,277 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E97316F327FA4D2D84F09F72A535DE7D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:56,277 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:56,278 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,278 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DA0740327CBD4EC1994A080E5DD4B3A3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:56,278 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:56,279 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DA0740327CBD4EC1994A080E5DD4B3A3
2016-05-17 20:45:56,279 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:56,279 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: DA0740327CBD4EC1994A080E5DD4B3A3
2016-05-17 20:45:56,279 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:56,287 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:56,287 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:56,288 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,295 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:56,295 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DA0740327CBD4EC1994A080E5DD4B3A3
2016-05-17 20:45:56,295 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A3F494A478E84615BDF9C31E63BA6EDC, ordinal: 1
2016-05-17 20:45:56,296 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3F494A478E84615BDF9C31E63BA6EDC, mem on start (mb), free: 317, total: 653, max: 672
2016-05-17 20:45:56,296 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,296 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:56,296 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:56,297 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,297 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:56,297 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:56,298 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:56,298 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:56,298 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:56,298 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:56,298 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:56,298 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DA0740327CBD4EC1994A080E5DD4B3A3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:56,299 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E97316F327FA4D2D84F09F72A535DE7D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:56,299 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:56,299 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,301 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:56,301 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:56,302 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E97316F327FA4D2D84F09F72A535DE7D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:56,302 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:56,302 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:56,302 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:56,302 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E97316F327FA4D2D84F09F72A535DE7D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:56,307 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,307 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:56,307 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:56,307 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:56,307 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,308 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517955973_0001_1_00_000000_0_10074, runDuration: 59609, ]
2016-05-17 20:45:56,310 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517955973_0001_1_00_000000_0_10074] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463517956310, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 20:45:56,310 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,310 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 20:45:56,311 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:56,311 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E97316F327FA4D2D84F09F72A535DE7D] #12 in 2ms
2016-05-17 20:45:56,312 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E97316F327FA4D2D84F09F72A535DE7D] #12, status:false, isInterrupted:false
2016-05-17 20:45:56,312 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 31FFDA18ACE24D1D8BDD6EAFAA2226C0
2016-05-17 20:45:56,312 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"], id: DA0740327CBD4EC1994A080E5DD4B3A3
2016-05-17 20:45:56,312 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-17 20:45:56,313 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 31FFDA18ACE24D1D8BDD6EAFAA2226C0, for 1 inputs
2016-05-17 20:45:56,314 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:56,314 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 20:45:56,314 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:56,315 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:56,315 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 20:45:56,315 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,315 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:56,316 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A3F494A478E84615BDF9C31E63BA6EDC, all 1 inputs ready in: 00:00:00.003
2016-05-17 20:45:56,318 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@394bff0a; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@2c5506e9
2016-05-17 20:45:56,322 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:56,322 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:56,323 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,323 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517955973_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:56,324 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,324 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3F494A478E84615BDF9C31E63BA6EDC, mem on close (mb), free: 315, total: 653, max: 672
2016-05-17 20:45:56,324 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,324 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:56,326 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:56,326 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:56,326 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:56,326 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,327 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:56,328 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517956316, Process_Duration=8, Process_End_Time=1463517956324, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 20:45:56,329 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:56,329 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:56,330 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,329 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0, startTime=1463517956272, finishTime=1463517956328, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956316, Process_Duration=8, Process_End_Time=1463517956324, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:56,330 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:56,330 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,330 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955973_0001_00_000002
2016-05-17 20:45:56,331 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:56,331 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_FINISHED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, taskId=task_1463517955973_0001_1_01_000000, startTime=1463517956272, finishTime=1463517956330, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1463517955973_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956316, Process_Duration=8, Process_End_Time=1463517956324, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:56,331 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:56,331 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:56,331 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] : 1
2016-05-17 20:45:56,331 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,332 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_FINISHED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, vertexId=vertex_1463517955973_0001_1_01, initRequestedTime=1463517956148, initedTime=1463517956157, startRequestedTime=1463517956166, startedTime=1463517956166, finishTime=1463517956331, timeTaken=165, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956316, Process_Duration=8, Process_End_Time=1463517956324, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463517956272, firstTasksToStart=[ task_1463517955973_0001_1_01_000000 ], lastTaskFinishTime=1463517956328, lastTasksToFinish=[ task_1463517955973_0001_1_01_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517955973_0001_1_01_000000 ], longestDurationTasks=[ task_1463517955973_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:56,333 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:56,333 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:56,333 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:56,333 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517955973_0001_00_000002, stoppedTime=1463517956333, exitStatus=0
2016-05-17 20:45:56,333 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:56,333 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:56,333 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517955973_0001_1
2016-05-17 20:45:56,334 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:56,334 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517955973_0001_1
2016-05-17 20:45:56,334 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: DA0740327CBD4EC1994A080E5DD4B3A3 for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,341 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517955973_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:56,341 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955973_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:56,341 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517955973_0001_00_000002, Exiting
2016-05-17 20:45:56,342 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517955973_0001_00_000002
2016-05-17 20:45:56,342 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517955973_0001_00_000002
2016-05-17 20:45:56,342 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517955973_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:56,342 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517955973_0001_00_000002 completed successfully
2016-05-17 20:45:56,344 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,345 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517955973_0001_1, queueSize=0
2016-05-17 20:45:56,346 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:56,346 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517955973_0001_1, startTime=1463517956146, finishTime=1463517956344, timeTaken=198, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1370488832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035912558, Process_Duration=10, Process_End_Time=2927035912568, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 20:45:56,346 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517955973_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:56,346 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955973_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:56,346 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:56 Completed Dag: dag_1463517955973_0001_1
2016-05-17 20:45:56,347 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517955973_0001_00_000002
2016-05-17 20:45:56,347 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517955973_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:56,347 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517955973_0001_1_post
2016-05-17 20:45:56,347 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,348 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:56,356 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463517955973_0001
2016-05-17 20:45:56,357 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:56,358 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:56,358 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:56,358 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:56,358 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:56,358 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:56,359 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:56,360 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:56,360 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:56,361 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:56,361 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:56,361 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:56,361 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:56,365 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001
2016-05-17 20:45:56,365 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3/_temporary
2016-05-17 20:45:56,366 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  completed in: 00:00.410
2016-05-17 20:45:56,433 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-17 20:45:56,456 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:56,456 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:56,490 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 20:45:56,498 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 20:45:56,498 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:56,498 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:56,498 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:56,512 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 20:45:56,518 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-17 20:45:56,563 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 20:45:56,564 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,564 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-17 20:45:56,564 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  parallel execution of steps is enabled: false
2016-05-17 20:45:56,564 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  executing total steps: 1
2016-05-17 20:45:56,564 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  allocating management threads: 1
2016-05-17 20:45:56,566 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount4/count4
2016-05-17 20:45:56,566 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:56,578 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 20:45:56,582 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:56,582 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:56,582 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:56,593 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001 doesn't exist and is created
2016-05-17 20:45:56,609 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:56,609 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:56,610 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001
2016-05-17 20:45:56,613 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517956582_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:56,623 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:56,624 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:56,627 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:56,627 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:56,627 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001/localmode-log-dir
2016-05-17 20:45:56,628 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001/localmode-log-dir/history.txt.appattempt_1463517956582_0001_000000, maxErrors=10
2016-05-17 20:45:56,628 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:56,628 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517956582_0001_000000, appSubmitTime=1463517956613, launchTime=1463517956613
2016-05-17 20:45:56,637 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:56,638 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42395
2016-05-17 20:45:56,638 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:56,640 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:56,640 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:56,641 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:56,644 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517956582_0001_000000, startTime=1463517956644
2016-05-17 20:45:56,644 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:56,644 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:56,710 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:56,710 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:56,710 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463517956582_0001, dagName=[B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4
2016-05-17 20:45:56,726 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4
2016-05-17 20:45:56,728 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517956582_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001/localmode-log-dir/dag_1463517956582_0001_1.dot
2016-05-17 20:45:56,728 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001/localmode-log-dir/dag_1463517956582_0001_1-tez-dag.pb.txt
2016-05-17 20:45:56,735 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517956582_0001_1
2016-05-17 20:45:56,735 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,735 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4
2016-05-17 20:45:56 Running Dag: dag_1463517956582_0001_1
2016-05-17 20:45:56,745 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517956582_0001_1, submitTime=1463517956726
2016-05-17 20:45:56,745 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:56,746 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,747 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:56,747 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517956582_0001_1, initTime=1463517956746
2016-05-17 20:45:56,747 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:56,747 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517956582_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:56,748 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_STARTED]: dagID=dag_1463517956582_0001_1, startTime=1463517956748
2016-05-17 20:45:56,748 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:56,748 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:56,748 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517956582_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:56,748 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463517956582_0001, dagName=[B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4
2016-05-17 20:45:56,749 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B3EC9DEBDCA44418BF89E7780E22AEEA : {50961CED08D749CF9E8EE1A94B2CA69A={InputName=50961CED08D749CF9E8EE1A94B2CA69A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:56,751 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 50961CED08D749CF9E8EE1A94B2CA69A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:56,751 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,751 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,752 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,752 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]. Starting root input initializers: 1
2016-05-17 20:45:56,752 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:56,753 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,754 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 50961CED08D749CF9E8EE1A94B2CA69A on vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,755 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:56,756 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,756 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,756 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,756 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D4CB83100CAD40BC965E4196415B5BF6, vertexId=vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:56,757 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D4CB83100CAD40BC965E4196415B5BF6, vertexId=vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,757 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:56,758 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 50961CED08D749CF9E8EE1A94B2CA69A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:56,758 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:56,759 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:56,759 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D4CB83100CAD40BC965E4196415B5BF6, vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:56,760 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D4CB83100CAD40BC965E4196415B5BF6, vertexId=vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,761 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:56,761 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 20:45:56,761 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 50961CED08D749CF9E8EE1A94B2CA69A on vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,762 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, vertexId=vertex_1463517956582_0001_1_01, initRequestedTime=1463517956753, initedTime=1463517956761, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:56,762 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:56,762 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:56,763 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {50961CED08D749CF9E8EE1A94B2CA69A=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:56,763 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] parallelism set to 1
2016-05-17 20:45:56,763 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:56,764 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517956582_0001_1_00, eventCount=1
2016-05-17 20:45:56,764 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, vertexId=vertex_1463517956582_0001_1_00, initRequestedTime=1463517956749, initedTime=1463517956764, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:56,764 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] with distanceFromRoot: 0
2016-05-17 20:45:56,764 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:56,765 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:56,765 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,765 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,766 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:56,766 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517956582_0001_1_00, startRequestedTime=1463517956765, startedTime=1463517956765
2016-05-17 20:45:56,766 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:56,767 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:56,767 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463517956582_0001, with dag id: dag_1463517956582_0001_1
2016-05-17 20:45:56,768 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_STARTED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, taskId=task_1463517956582_0001_1_00_000000, scheduledTime=1463517956767, launchTime=1463517956767
2016-05-17 20:45:56,768 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:56,769 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517956582_0001_1_00 for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] numStartedSources: 1 numSources: 1
2016-05-17 20:45:56,769 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] with distanceFromRoot: 1
2016-05-17 20:45:56,769 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,769 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B3EC9DEBDCA44418BF89E7780E22AEEA in vertex: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,772 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CEA5F0E3B8F5489594732E8C4C022C3D with 1 source tasks and 1 pending tasks
2016-05-17 20:45:56,773 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517956582_0001_1_01, startRequestedTime=1463517956769, startedTime=1463517956769
2016-05-17 20:45:56,773 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:56,773 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:56,774 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4, VertexName: B3EC9DEBDCA44418BF89E7780E22AEEA, VertexParallelism: 1, TaskAttemptID:attempt_1463517956582_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=50961CED08D749CF9E8EE1A94B2CA69A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CEA5F0E3B8F5489594732E8C4C022C3D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:56,774 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:56,774 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:56,778 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:56,779 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:56,779 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:56,780 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517956582_0001_00_000001
2016-05-17 20:45:56,781 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:56,783 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517956582_0001_00_000001
2016-05-17 20:45:56,783 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:56,784 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:56,784 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:56,784 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517956582_0001_00_000001, launchTime=1463517956783
2016-05-17 20:45:56,784 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517956582_0001_00_000001
2016-05-17 20:45:56,785 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517956582_0001_1_00_000000_0] to container: [container_1463517956582_0001_00_000001]
2016-05-17 20:45:56,785 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:56,785 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517956582_0001_00_000001 given task: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,785 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517956582_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,786 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517956582_0001_1_00_000000_0] started. Is using containerId: [container_1463517956582_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,786 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,787 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,787 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:56,787 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:56,787 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:56,787 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:56,787 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:56,787 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4, VertexName: B3EC9DEBDCA44418BF89E7780E22AEEA, VertexParallelism: 1, TaskAttemptID:attempt_1463517956582_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=50961CED08D749CF9E8EE1A94B2CA69A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CEA5F0E3B8F5489594732E8C4C022C3D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:56,787 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:56,788 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:56,788 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0, startTime=1463517956786, containerId=container_1463517956582_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517956582_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:56,788 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,788 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:56,788 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,788 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:56,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CEA5F0E3B8F5489594732E8C4C022C3D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:56,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=50961CED08D749CF9E8EE1A94B2CA69A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:56,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:56,792 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:56,792 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517956582_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:56,794 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,794 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,794 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,794 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:56,797 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:56,797 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:56,798 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,798 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,801 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:56,801 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,801 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,803 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B3EC9DEBDCA44418BF89E7780E22AEEA, ordinal: 0
2016-05-17 20:45:56,803 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3EC9DEBDCA44418BF89E7780E22AEEA, mem on start (mb), free: 226, total: 653, max: 672
2016-05-17 20:45:56,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:56,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:56,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:56,803 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:56,803 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:56,803 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:56,804 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:56,804 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:56,804 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:56,804 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 50961CED08D749CF9E8EE1A94B2CA69A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:56,804 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CEA5F0E3B8F5489594732E8C4C022C3D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:56,804 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:56,804 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 50961CED08D749CF9E8EE1A94B2CA69A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:56,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:56,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:56,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,805 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:56,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:56,805 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,805 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:56,808 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3bfd4ff0
2016-05-17 20:45:56,808 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,809 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,809 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: B00C424EED6E435BA56DD6DF2A1DB36B
2016-05-17 20:45:56,809 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] B00C424EED6E435BA56DD6DF2A1DB36B
2016-05-17 20:45:56,810 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3bfd4ff0
2016-05-17 20:45:56,810 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:56,810 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:56,811 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@52950d16; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@378e49b5; comparator=cascading.tuple.hadoop.util.TupleComparator@7e432a13; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:56,812 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:56,813 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:56,813 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:56,835 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:56,835 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:56,835 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:56,835 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:56,835 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:56,836 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,836 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:56,837 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B3EC9DEBDCA44418BF89E7780E22AEEA, all 1 inputs ready in: 00:00:00.001
2016-05-17 20:45:56,843 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,843 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3EC9DEBDCA44418BF89E7780E22AEEA, mem on close (mb), free: 322, total: 654, max: 672
2016-05-17 20:45:56,843 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:56,843 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:56,844 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 20:45:56,844 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 20:45:56,848 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,851 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:56,852 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517956582_0001_1_00, eventCount=1
2016-05-17 20:45:56,852 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517956837, Process_Duration=6, Process_End_Time=1463517956843, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 20:45:56,852 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:56,852 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:56,852 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0, startTime=1463517956786, finishTime=1463517956852, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956837, Process_Duration=6, Process_End_Time=1463517956843, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:56,853 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,853 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:56,853 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,853 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517956582_0001_00_000001
2016-05-17 20:45:56,854 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:56,854 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:56,854 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_FINISHED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, taskId=task_1463517956582_0001_1_00_000000, startTime=1463517956786, finishTime=1463517956853, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1463517956582_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956837, Process_Duration=6, Process_End_Time=1463517956843, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:56,855 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:56,855 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] : 1
2016-05-17 20:45:56,855 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,856 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_FINISHED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, vertexId=vertex_1463517956582_0001_1_00, initRequestedTime=1463517956749, initedTime=1463517956764, startRequestedTime=1463517956765, startedTime=1463517956765, finishTime=1463517956855, timeTaken=90, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956837, Process_Duration=6, Process_End_Time=1463517956843, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463517956786, firstTasksToStart=[ task_1463517956582_0001_1_00_000000 ], lastTaskFinishTime=1463517956852, lastTasksToFinish=[ task_1463517956582_0001_1_00_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517956582_0001_1_00_000000 ], longestDurationTasks=[ task_1463517956582_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:56,857 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:56,857 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:56,857 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517956582_0001_00_000001, stoppedTime=1463517956857, exitStatus=0
2016-05-17 20:45:56,857 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:56,857 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] attempt: attempt_1463517956582_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:56,857 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,857 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,858 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:56,858 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_STARTED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, taskId=task_1463517956582_0001_1_01_000000, scheduledTime=1463517956858, launchTime=1463517956858
2016-05-17 20:45:56,858 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:56,858 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517956582_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:56,858 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:56,858 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,859 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:56,859 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:56,859 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:56,860 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4, VertexName: CEA5F0E3B8F5489594732E8C4C022C3D, VertexParallelism: 1, TaskAttemptID:attempt_1463517956582_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D4CB83100CAD40BC965E4196415B5BF6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:56,860 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:56,860 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:56,860 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517956582_0001_00_000002
2016-05-17 20:45:56,860 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:56,862 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517956582_0001_00_000002
2016-05-17 20:45:56,862 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:56,862 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:56,862 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:56,862 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517956582_0001_00_000002, launchTime=1463517956862
2016-05-17 20:45:56,863 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517956582_0001_00_000002
2016-05-17 20:45:56,863 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517956582_0001_1_01_000000_0] to container: [container_1463517956582_0001_00_000002]
2016-05-17 20:45:56,863 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:56,864 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517956582_0001_00_000002 given task: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,864 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517956582_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,864 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517956582_0001_1_01_000000_0] started. Is using containerId: [container_1463517956582_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:56,864 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:56,864 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517956582_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:56,864 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0, startTime=1463517956864, containerId=container_1463517956582_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517956582_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:56,865 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:56,865 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517956582_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:56,865 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:56,865 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517956582_0001_00_000001, Exiting
2016-05-17 20:45:56,865 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517956582_0001_00_000001
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,865 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517956582_0001_00_000001
2016-05-17 20:45:56,865 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,865 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,865 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:56,865 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:56,866 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:56,866 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:56,866 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:56,866 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4, VertexName: CEA5F0E3B8F5489594732E8C4C022C3D, VertexParallelism: 1, TaskAttemptID:attempt_1463517956582_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D4CB83100CAD40BC965E4196415B5BF6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:56,866 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:56,867 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:56,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,868 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517956582_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:56,867 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517956582_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:56,868 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517956582_0001_00_000001 completed successfully
2016-05-17 20:45:56,868 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:56,869 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:56,869 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,870 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:56,871 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:56,871 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:56,871 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,872 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,872 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,872 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D4CB83100CAD40BC965E4196415B5BF6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:56,873 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:56,873 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D4CB83100CAD40BC965E4196415B5BF6
2016-05-17 20:45:56,873 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:56,873 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D4CB83100CAD40BC965E4196415B5BF6
2016-05-17 20:45:56,875 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:56,881 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:56,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D4CB83100CAD40BC965E4196415B5BF6
2016-05-17 20:45:56,883 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CEA5F0E3B8F5489594732E8C4C022C3D, ordinal: 1
2016-05-17 20:45:56,883 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CEA5F0E3B8F5489594732E8C4C022C3D, mem on start (mb), free: 316, total: 654, max: 672
2016-05-17 20:45:56,883 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,884 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:56,884 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:56,884 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:56,885 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:56,885 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:56,885 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:56,886 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:56,886 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:56,887 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:56,887 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B3EC9DEBDCA44418BF89E7780E22AEEA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:56,887 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D4CB83100CAD40BC965E4196415B5BF6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:56,888 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:56,888 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B3EC9DEBDCA44418BF89E7780E22AEEA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:56,888 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,889 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:56,889 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:56,889 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:56,890 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:56,890 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B3EC9DEBDCA44418BF89E7780E22AEEA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:56,890 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:56,891 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:56,891 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:56,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:56,893 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,894 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517956582_0001_1_00_000000_0_10079, runDuration: 53117, ]
2016-05-17 20:45:56,896 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 20:45:56,896 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517956582_0001_1_00_000000_0_10079] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463517956896, TimeTaken=1, Rate=0.44 MB/s
2016-05-17 20:45:56,897 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,898 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:56,898 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B3EC9DEBDCA44418BF89E7780E22AEEA] #13 in 3ms
2016-05-17 20:45:56,899 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: B00C424EED6E435BA56DD6DF2A1DB36B
2016-05-17 20:45:56,899 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"], id: D4CB83100CAD40BC965E4196415B5BF6
2016-05-17 20:45:56,900 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-17 20:45:56,900 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] B00C424EED6E435BA56DD6DF2A1DB36B, for 1 inputs
2016-05-17 20:45:56,899 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B3EC9DEBDCA44418BF89E7780E22AEEA] #13, status:false, isInterrupted:false
2016-05-17 20:45:56,901 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:56,902 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 20:45:56,903 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:56,903 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:56,903 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 20:45:56,904 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CEA5F0E3B8F5489594732E8C4C022C3D, all 1 inputs ready in: 00:00:00.004
2016-05-17 20:45:56,904 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,906 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1f3ad2aa; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@4edca6ea
2016-05-17 20:45:56,906 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:56,909 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:56,910 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:56,910 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,910 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517956582_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:56,911 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,911 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CEA5F0E3B8F5489594732E8C4C022C3D, mem on close (mb), free: 313, total: 654, max: 672
2016-05-17 20:45:56,911 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,912 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:56,913 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:56,913 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:56,913 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:56,914 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,915 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:56,915 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517956904, Process_Duration=7, Process_End_Time=1463517956911, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 20:45:56,916 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:56,916 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0, startTime=1463517956864, finishTime=1463517956915, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956904, Process_Duration=7, Process_End_Time=1463517956911, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:56,916 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:56,916 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,916 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:56,917 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:56,917 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,917 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:56,917 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_FINISHED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, taskId=task_1463517956582_0001_1_01_000000, startTime=1463517956864, finishTime=1463517956916, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1463517956582_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956904, Process_Duration=7, Process_End_Time=1463517956911, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:56,917 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:56,917 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] : 1
2016-05-17 20:45:56,917 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,919 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:56,917 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517956582_0001_00_000002
2016-05-17 20:45:56,920 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_FINISHED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, vertexId=vertex_1463517956582_0001_1_01, initRequestedTime=1463517956753, initedTime=1463517956761, startRequestedTime=1463517956769, startedTime=1463517956769, finishTime=1463517956918, timeTaken=149, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956904, Process_Duration=7, Process_End_Time=1463517956911, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463517956864, firstTasksToStart=[ task_1463517956582_0001_1_01_000000 ], lastTaskFinishTime=1463517956915, lastTasksToFinish=[ task_1463517956582_0001_1_01_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517956582_0001_1_01_000000 ], longestDurationTasks=[ task_1463517956582_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:56,920 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:56,920 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:56,920 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517956582_0001_00_000002, stoppedTime=1463517956920, exitStatus=0
2016-05-17 20:45:56,921 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:56,921 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:56,921 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517956582_0001_1
2016-05-17 20:45:56,921 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517956582_0001_1
2016-05-17 20:45:56,921 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,921 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D4CB83100CAD40BC965E4196415B5BF6 for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,922 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:56,929 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517956582_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:56,929 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517956582_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:56,929 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517956582_0001_00_000002, Exiting
2016-05-17 20:45:56,929 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517956582_0001_00_000002
2016-05-17 20:45:56,929 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517956582_0001_00_000002
2016-05-17 20:45:56,930 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517956582_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:56,930 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517956582_0001_00_000002 completed successfully
2016-05-17 20:45:56,931 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517956582_0001_1, queueSize=0
2016-05-17 20:45:56,931 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:56,932 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517956582_0001_1, startTime=1463517956748, finishTime=1463517956928, timeTaken=180, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1372585984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035913741, Process_Duration=13, Process_End_Time=2927035913754, Read_Duration=6, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 20:45:56,932 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517956582_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:56,933 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517956582_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:56,933 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517956582_0001_00_000002
2016-05-17 20:45:56,935 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:56 Completed Dag: dag_1463517956582_0001_1
2016-05-17 20:45:56,937 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517956582_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:56,940 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517956582_0001_1_post
2016-05-17 20:45:56,940 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,940 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:56,940 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463517956582_0001
2016-05-17 20:45:56,942 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:56,942 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:56,942 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:56,942 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:56,943 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:56,944 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:56,944 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:56,944 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:56,944 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:56,944 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:56,945 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:56,945 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:56,948 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:56,950 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001
2016-05-17 20:45:56,950 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4/_temporary
2016-05-17 20:45:56,951 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [count]  completed in: 00:00.387
2016-05-17 20:45:57,019 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-17 20:45:57,041 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:57,041 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:57,071 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 20:45:57,078 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 20:45:57,079 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:57,079 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:57,080 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:57,094 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:57,094 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-17 20:45:57,103 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:57,103 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,103 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-17 20:45:57,104 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:57,104 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:57,104 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:57,105 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testFilter/filter
2016-05-17 20:45:57,105 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:57,117 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:57,121 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:57,122 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:57,122 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:57,131 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001 doesn't exist and is created
2016-05-17 20:45:57,146 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:57,146 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:57,146 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001
2016-05-17 20:45:57,149 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517957121_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:57,159 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:57,160 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:57,161 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:57,161 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:57,161 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001/localmode-log-dir
2016-05-17 20:45:57,162 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001/localmode-log-dir/history.txt.appattempt_1463517957121_0001_000000, maxErrors=10
2016-05-17 20:45:57,162 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:57,162 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517957121_0001_000000, appSubmitTime=1463517957149, launchTime=1463517957149
2016-05-17 20:45:57,165 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:57,165 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:57,166 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:57,167 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:57,170 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51116
2016-05-17 20:45:57,171 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:57,173 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517957121_0001_000000, startTime=1463517957173
2016-05-17 20:45:57,173 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:57,173 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:57,246 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:57,246 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:57,246 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463517957121_0001, dagName=[CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter
2016-05-17 20:45:57,261 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter
2016-05-17 20:45:57,263 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517957121_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001/localmode-log-dir/dag_1463517957121_0001_1.dot
2016-05-17 20:45:57,263 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001/localmode-log-dir/dag_1463517957121_0001_1-tez-dag.pb.txt
2016-05-17 20:45:57,268 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517957121_0001_1
2016-05-17 20:45:57,268 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,268 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter
2016-05-17 20:45:57 Running Dag: dag_1463517957121_0001_1
2016-05-17 20:45:57,278 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:57,279 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517957121_0001_1, submitTime=1463517957262
2016-05-17 20:45:57,279 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E5134DB3CCA440B1BC64C1CBE0ECF57E
2016-05-17 20:45:57,279 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:57,280 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:57,280 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517957121_0001_1, initTime=1463517957279
2016-05-17 20:45:57,280 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957121_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:57,281 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_STARTED]: dagID=dag_1463517957121_0001_1, startTime=1463517957281
2016-05-17 20:45:57,281 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:57,281 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:57,281 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957121_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:57,282 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E5134DB3CCA440B1BC64C1CBE0ECF57E : {96B49CDB7AA747DF833FC44A935DBF0D={InputName=96B49CDB7AA747DF833FC44A935DBF0D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:57,282 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 96B49CDB7AA747DF833FC44A935DBF0D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:57,282 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463517957121_0001, dagName=[CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter
2016-05-17 20:45:57,282 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,282 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,282 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,282 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]. Starting root input initializers: 1
2016-05-17 20:45:57,283 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:57,284 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 96B49CDB7AA747DF833FC44A935DBF0D on vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,286 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:57,286 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 96B49CDB7AA747DF833FC44A935DBF0D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:57,287 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:57,287 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:57,288 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:57,289 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 96B49CDB7AA747DF833FC44A935DBF0D on vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,289 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:57,289 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {96B49CDB7AA747DF833FC44A935DBF0D=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:57,289 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] parallelism set to 1
2016-05-17 20:45:57,289 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:57,290 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517957121_0001_1_00, eventCount=1
2016-05-17 20:45:57,290 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,290 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FA491E0CA247478F8B6C06CEAB003090, vertexId=vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:57,290 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FA491E0CA247478F8B6C06CEAB003090, vertexId=vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,292 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FA491E0CA247478F8B6C06CEAB003090, vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:57,292 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FA491E0CA247478F8B6C06CEAB003090, vertexId=vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,293 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, vertexId=vertex_1463517957121_0001_1_00, initRequestedTime=1463517957282, initedTime=1463517957293, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:57,293 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:57,294 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] with distanceFromRoot: 0
2016-05-17 20:45:57,294 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:57,294 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E5134DB3CCA440B1BC64C1CBE0ECF57E
2016-05-17 20:45:57,294 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,295 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:57,295 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517957121_0001_1_00, startRequestedTime=1463517957294, startedTime=1463517957294
2016-05-17 20:45:57,295 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:57,296 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:57,296 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:TASK_STARTED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, taskId=task_1463517957121_0001_1_00_000000, scheduledTime=1463517957296, launchTime=1463517957296
2016-05-17 20:45:57,296 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957121_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:57,297 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,297 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter, VertexName: E5134DB3CCA440B1BC64C1CBE0ECF57E, VertexParallelism: 1, TaskAttemptID:attempt_1463517957121_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=96B49CDB7AA747DF833FC44A935DBF0D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FA491E0CA247478F8B6C06CEAB003090, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:57,297 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957121_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:57,298 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:57,299 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:57,299 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:57,299 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:57,300 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517957121_0001_00_000001
2016-05-17 20:45:57,300 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:57,302 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517957121_0001_00_000001
2016-05-17 20:45:57,302 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:57,303 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:57,303 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517957121_0001_00_000001, launchTime=1463517957303
2016-05-17 20:45:57,303 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:57,304 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957121_0001_00_000001
2016-05-17 20:45:57,304 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517957121_0001_1_00_000000_0] to container: [container_1463517957121_0001_00_000001]
2016-05-17 20:45:57,304 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:57,305 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517957121_0001, with dag id: dag_1463517957121_0001_1
2016-05-17 20:45:57,305 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517957121_0001_00_000001 given task: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,306 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957121_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,307 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517957121_0001_1_00_000000_0] started. Is using containerId: [container_1463517957121_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:57 Starting to run new task attempt: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,307 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,307 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,307 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:57,307 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0, startTime=1463517957306, containerId=container_1463517957121_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517957121_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:57,308 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957121_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:57,307 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:57,308 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:57,308 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:57,309 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:57,309 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:57,309 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter, VertexName: E5134DB3CCA440B1BC64C1CBE0ECF57E, VertexParallelism: 1, TaskAttemptID:attempt_1463517957121_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=96B49CDB7AA747DF833FC44A935DBF0D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FA491E0CA247478F8B6C06CEAB003090, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:57,309 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:57,309 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957121_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:57,310 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,310 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:57,310 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,311 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517957121_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:57,311 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,312 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=96B49CDB7AA747DF833FC44A935DBF0D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:57,312 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:57,313 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,313 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,314 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:57,316 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:57,317 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,317 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,317 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FA491E0CA247478F8B6C06CEAB003090, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:57,318 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:57,318 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FA491E0CA247478F8B6C06CEAB003090
2016-05-17 20:45:57,318 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:57,318 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FA491E0CA247478F8B6C06CEAB003090
2016-05-17 20:45:57,320 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E5134DB3CCA440B1BC64C1CBE0ECF57E, ordinal: 0
2016-05-17 20:45:57,320 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5134DB3CCA440B1BC64C1CBE0ECF57E, mem on start (mb), free: 241, total: 654, max: 672
2016-05-17 20:45:57,320 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,321 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:57,321 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:57,330 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:57,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FA491E0CA247478F8B6C06CEAB003090
2016-05-17 20:45:57,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:57,330 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:57,330 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:57,330 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:57,331 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:57,331 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:57,331 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:57,331 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:57,331 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 96B49CDB7AA747DF833FC44A935DBF0D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:57,331 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FA491E0CA247478F8B6C06CEAB003090, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:57,331 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:57,331 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 96B49CDB7AA747DF833FC44A935DBF0D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:57,332 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:57,332 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:57,332 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,332 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:57,332 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,333 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:57,333 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,336 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,337 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,337 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"], id: FA491E0CA247478F8B6C06CEAB003090
2016-05-17 20:45:57,337 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-17 20:45:57,337 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,337 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:57,338 INFO  input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 20:45:57,338 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:57,338 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2e5524e
2016-05-17 20:45:57,339 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2e5524e
2016-05-17 20:45:57,340 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:57,340 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:57,340 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E5134DB3CCA440B1BC64C1CBE0ECF57E, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:57,342 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:57,342 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:57,343 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,343 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517957121_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:57,344 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,344 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5134DB3CCA440B1BC64C1CBE0ECF57E, mem on close (mb), free: 240, total: 654, max: 672
2016-05-17 20:45:57 Completed running task attempt: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,345 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:57,346 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=3]]
2016-05-17 20:45:57,346 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0, startTime=1463517957306, finishTime=1463517957346, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 20:45:57,346 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:57,347 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957121_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:57,347 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:57,347 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,347 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:57,347 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,348 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957121_0001_00_000001
2016-05-17 20:45:57,348 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:TASK_FINISHED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, taskId=task_1463517957121_0001_1_00_000000, startTime=1463517957306, finishTime=1463517957347, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1463517957121_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 20:45:57,348 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:57,348 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957121_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:57,349 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] : 1
2016-05-17 20:45:57,349 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,350 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:57,350 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:VERTEX_FINISHED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, vertexId=vertex_1463517957121_0001_1_00, initRequestedTime=1463517957282, initedTime=1463517957293, startRequestedTime=1463517957294, startedTime=1463517957294, finishTime=1463517957349, timeTaken=55, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3, vertexStats=firstTaskStartTime=1463517957306, firstTasksToStart=[ task_1463517957121_0001_1_00_000000 ], lastTaskFinishTime=1463517957346, lastTasksToFinish=[ task_1463517957121_0001_1_00_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517957121_0001_1_00_000000 ], longestDurationTasks=[ task_1463517957121_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:57,350 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:57,350 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:57,350 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517957121_0001_00_000001, stoppedTime=1463517957350, exitStatus=0
2016-05-17 20:45:57,350 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:57,351 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:57,351 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:57,351 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517957121_0001_1
2016-05-17 20:45:57,351 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517957121_0001_1
2016-05-17 20:45:57,351 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: FA491E0CA247478F8B6C06CEAB003090 for vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,358 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517957121_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:57,359 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957121_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:57,359 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517957121_0001_00_000001, Exiting
2016-05-17 20:45:57,359 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517957121_0001_00_000001
2016-05-17 20:45:57,359 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517957121_0001_00_000001
2016-05-17 20:45:57,360 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517957121_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:57,360 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517957121_0001_00_000001 completed successfully
2016-05-17 20:45:57,361 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517957121_0001_1, queueSize=0
2016-05-17 20:45:57,361 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:57,362 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517957121_0001_1, startTime=1463517957281, finishTime=1463517957358, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 20:45:57,362 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517957121_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:57,362 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957121_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:57,363 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517957121_0001_00_000001
2016-05-17 20:45:57,363 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:57 Completed Dag: dag_1463517957121_0001_1
2016-05-17 20:45:57,364 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517957121_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:57,365 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517957121_0001_1_post
2016-05-17 20:45:57,365 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,365 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:57,368 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463517957121_0001
2016-05-17 20:45:57,370 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:57,370 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:57,370 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:57,370 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:57,370 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:57,371 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:57,371 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:57,371 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:57,372 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:57,372 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:57,372 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:57,373 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:57,373 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:57,376 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001
2016-05-17 20:45:57,377 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter/_temporary
2016-05-17 20:45:57,377 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.274
2016-05-17 20:45:57,410 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-17 20:45:57,435 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:57,435 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:57,458 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:57,458 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:57,471 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 20:45:57,486 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-17 20:45:57,487 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:57,487 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:57,488 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:57,503 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:57,518 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple
2016-05-17 20:45:57,563 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:57,564 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,565 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-17 20:45:57,565 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-17 20:45:57,565 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-17 20:45:57,566 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-17 20:45:57,567 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testSimple/simple
2016-05-17 20:45:57,567 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:57,578 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:57,582 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:57,583 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:57,583 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:57,592 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001 doesn't exist and is created
2016-05-17 20:45:57,607 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:57,607 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:57,608 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001
2016-05-17 20:45:57,610 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517957583_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:57,619 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:57,620 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:57,621 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:57,622 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:57,622 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001/localmode-log-dir
2016-05-17 20:45:57,622 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001/localmode-log-dir/history.txt.appattempt_1463517957583_0001_000000, maxErrors=10
2016-05-17 20:45:57,622 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:57,623 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517957583_0001_000000, appSubmitTime=1463517957609, launchTime=1463517957610
2016-05-17 20:45:57,625 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:57,628 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:57,630 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:57,630 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:57,631 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42671
2016-05-17 20:45:57,632 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:57,636 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517957583_0001_000000, startTime=1463517957636
2016-05-17 20:45:57,636 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:57,636 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:57,707 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:57,707 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:57,707 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463517957583_0001, dagName=[D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple
2016-05-17 20:45:57,723 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple
2016-05-17 20:45:57,725 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517957583_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001/localmode-log-dir/dag_1463517957583_0001_1.dot
2016-05-17 20:45:57,725 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001/localmode-log-dir/dag_1463517957583_0001_1-tez-dag.pb.txt
2016-05-17 20:45:57,736 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517957583_0001_1
2016-05-17 20:45:57,736 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,737 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple
2016-05-17 20:45:57 Running Dag: dag_1463517957583_0001_1
2016-05-17 20:45:57,749 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:57,749 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517957583_0001_1, submitTime=1463517957723
2016-05-17 20:45:57,750 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,750 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:57,751 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:57,752 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517957583_0001_1, initTime=1463517957750
2016-05-17 20:45:57,752 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957583_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:57,752 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_STARTED]: dagID=dag_1463517957583_0001_1, startTime=1463517957752
2016-05-17 20:45:57,752 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:57,752 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:57,753 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957583_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:57,753 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CE088C9F6B3E48728F84F5607D3748C4 : {E9E5621A47A145F1B219FD77E7B7324E={InputName=E9E5621A47A145F1B219FD77E7B7324E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:57,753 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E9E5621A47A145F1B219FD77E7B7324E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:57,753 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463517957583_0001, dagName=[D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple
2016-05-17 20:45:57,753 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,754 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,754 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,754 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]. Starting root input initializers: 1
2016-05-17 20:45:57,755 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:57,755 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,758 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:57,758 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,758 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,758 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,758 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A4B7AEA4155F4E6FA46F24EE0D2D08B9, vertexId=vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:57,759 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A4B7AEA4155F4E6FA46F24EE0D2D08B9, vertexId=vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,761 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A4B7AEA4155F4E6FA46F24EE0D2D08B9, vertexName=D32E1B14CEFD419D9F85225A98723848, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:57,762 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A4B7AEA4155F4E6FA46F24EE0D2D08B9, vertexId=vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,761 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E9E5621A47A145F1B219FD77E7B7324E on vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,765 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:57,765 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D32E1B14CEFD419D9F85225A98723848, vertexId=vertex_1463517957583_0001_1_01, initRequestedTime=1463517957755, initedTime=1463517957763, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:57,765 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:57,766 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:57,767 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E9E5621A47A145F1B219FD77E7B7324E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:57,767 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:57,767 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:57,768 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:57,769 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E9E5621A47A145F1B219FD77E7B7324E on vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,769 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:57,769 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E9E5621A47A145F1B219FD77E7B7324E=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:57,769 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] parallelism set to 1
2016-05-17 20:45:57,769 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:57,770 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517957583_0001_1_00, eventCount=1
2016-05-17 20:45:57,770 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:57,770 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517957583_0001, with dag id: dag_1463517957583_0001_1
2016-05-17 20:45:57,771 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, vertexId=vertex_1463517957583_0001_1_00, initRequestedTime=1463517957753, initedTime=1463517957770, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:57,772 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] with distanceFromRoot: 0
2016-05-17 20:45:57,772 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:57,772 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,773 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,773 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:57,773 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517957583_0001_1_00, startRequestedTime=1463517957772, startedTime=1463517957772
2016-05-17 20:45:57,774 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:57,775 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:57,775 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_STARTED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, taskId=task_1463517957583_0001_1_00_000000, scheduledTime=1463517957775, launchTime=1463517957775
2016-05-17 20:45:57,775 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:57,776 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517957583_0001_1_00 for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] numStartedSources: 1 numSources: 1
2016-05-17 20:45:57,776 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] with distanceFromRoot: 1
2016-05-17 20:45:57,777 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,778 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CE088C9F6B3E48728F84F5607D3748C4 in vertex: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,778 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D32E1B14CEFD419D9F85225A98723848 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:57,778 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:57,779 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517957583_0001_1_01, startRequestedTime=1463517957777, startedTime=1463517957777
2016-05-17 20:45:57,779 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:57,779 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple, VertexName: CE088C9F6B3E48728F84F5607D3748C4, VertexParallelism: 1, TaskAttemptID:attempt_1463517957583_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E9E5621A47A145F1B219FD77E7B7324E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D32E1B14CEFD419D9F85225A98723848, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:57,779 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:57,779 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:57,780 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:57,780 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:57,781 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:57,781 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517957583_0001_00_000001
2016-05-17 20:45:57,781 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:57,783 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517957583_0001_00_000001
2016-05-17 20:45:57,783 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:57,784 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:57,784 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:57,784 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517957583_0001_00_000001, launchTime=1463517957784
2016-05-17 20:45:57,785 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957583_0001_00_000001
2016-05-17 20:45:57,785 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517957583_0001_1_00_000000_0] to container: [container_1463517957583_0001_00_000001]
2016-05-17 20:45:57,786 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:57,786 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517957583_0001_1_00_000000_0] started. Is using containerId: [container_1463517957583_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:57,786 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0, startTime=1463517957786, containerId=container_1463517957583_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517957583_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:57,786 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:57,786 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:57,787 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:57,787 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517957583_0001_00_000001 given task: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,787 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957583_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57 Starting to run new task attempt: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,788 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,788 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,788 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:57,788 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:57,788 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:57,788 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:57,789 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:57,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple, VertexName: CE088C9F6B3E48728F84F5607D3748C4, VertexParallelism: 1, TaskAttemptID:attempt_1463517957583_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E9E5621A47A145F1B219FD77E7B7324E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D32E1B14CEFD419D9F85225A98723848, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:57,789 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:57,789 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,790 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:57,790 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,792 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517957583_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:57,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E9E5621A47A145F1B219FD77E7B7324E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:57,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D32E1B14CEFD419D9F85225A98723848, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:57,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,794 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:57,794 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:57,795 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,795 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,795 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,797 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:57,797 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,798 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,798 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:57,798 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:57,798 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,805 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CE088C9F6B3E48728F84F5607D3748C4, ordinal: 0
2016-05-17 20:45:57,805 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE088C9F6B3E48728F84F5607D3748C4, mem on start (mb), free: 371, total: 655, max: 672
2016-05-17 20:45:57,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:57,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:57,807 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:57,807 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:57,807 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:57,808 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:57,808 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:57,809 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:57,809 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:57,809 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E9E5621A47A145F1B219FD77E7B7324E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:57,810 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D32E1B14CEFD419D9F85225A98723848, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:57,810 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:57,810 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,810 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:57,811 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,811 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E9E5621A47A145F1B219FD77E7B7324E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:57,811 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:57,812 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:57,812 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:57,812 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,812 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:57,813 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4b28f983
2016-05-17 20:45:57,814 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4b28f983
2016-05-17 20:45:57,814 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:57,814 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:57,816 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,817 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,817 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: AF0DE59E97534EF6B52278B068B16535
2016-05-17 20:45:57,817 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] AF0DE59E97534EF6B52278B068B16535
2016-05-17 20:45:57,820 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3defad77; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@42657f65; comparator=cascading.tuple.hadoop.util.TupleComparator@5793a045; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:57,820 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:57,821 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:57,821 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:57,835 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:57,836 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:57,836 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:57,836 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:57,837 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:57,837 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,838 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:57,838 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CE088C9F6B3E48728F84F5607D3748C4, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:57,840 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,841 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE088C9F6B3E48728F84F5607D3748C4, mem on close (mb), free: 269, total: 655, max: 672
2016-05-17 20:45:57,841 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:57,841 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:57,841 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 20:45:57,842 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 20:45:57,844 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:57 Completed running task attempt: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,848 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:57,849 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517957583_0001_1_00, eventCount=1
2016-05-17 20:45:57,849 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:57,850 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0, startTime=1463517957786, finishTime=1463517957849, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957838, Process_Duration=2, Process_End_Time=1463517957840, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:57,850 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:57,850 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:57,850 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,850 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:57,851 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_FINISHED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, taskId=task_1463517957583_0001_1_00_000000, startTime=1463517957786, finishTime=1463517957850, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1463517957583_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957838, Process_Duration=2, Process_End_Time=1463517957840, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:57,851 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:57,851 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] : 1
2016-05-17 20:45:57,851 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,851 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517957838, Process_Duration=2, Process_End_Time=1463517957840, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 20:45:57,853 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_FINISHED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, vertexId=vertex_1463517957583_0001_1_00, initRequestedTime=1463517957753, initedTime=1463517957770, startRequestedTime=1463517957772, startedTime=1463517957772, finishTime=1463517957851, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957838, Process_Duration=2, Process_End_Time=1463517957840, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463517957786, firstTasksToStart=[ task_1463517957583_0001_1_00_000000 ], lastTaskFinishTime=1463517957849, lastTasksToFinish=[ task_1463517957583_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517957583_0001_1_00_000000 ], longestDurationTasks=[ task_1463517957583_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:57,853 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:57,855 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:57,854 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:57,855 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,855 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957583_0001_00_000001
2016-05-17 20:45:57,855 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:57,856 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517957583_0001_00_000001, stoppedTime=1463517957856, exitStatus=0
2016-05-17 20:45:57,856 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:57,856 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] attempt: attempt_1463517957583_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:57,856 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,857 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,857 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:57,857 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517957583_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:57,856 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:57,858 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:57,857 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_STARTED]: vertexName=D32E1B14CEFD419D9F85225A98723848, taskId=task_1463517957583_0001_1_01_000000, scheduledTime=1463517957857, launchTime=1463517957857
2016-05-17 20:45:57,858 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:57,856 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(221)) - No more tasks for container with id : container_1463517957583_0001_00_000001. Asking it to die
2016-05-17 20:45:57,858 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957583_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:57,858 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517957583_0001_00_000001, Exiting
2016-05-17 20:45:57,859 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517957583_0001_00_000001
2016-05-17 20:45:57,859 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517957583_0001_00_000001
2016-05-17 20:45:57,858 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,859 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:57,860 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple, VertexName: D32E1B14CEFD419D9F85225A98723848, VertexParallelism: 1, TaskAttemptID:attempt_1463517957583_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CE088C9F6B3E48728F84F5607D3748C4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A4B7AEA4155F4E6FA46F24EE0D2D08B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:57,860 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:57,860 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:57,861 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517957583_0001_00_000002
2016-05-17 20:45:57,861 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:57,861 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517957583_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:57,862 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517957583_0001_00_000001 completed successfully
2016-05-17 20:45:57,862 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1463517957583_0001_00_000002
2016-05-17 20:45:57,863 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:57,863 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:57,865 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:57,865 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:57,865 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517957583_0001_00_000002, launchTime=1463517957865
2016-05-17 20:45:57,866 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957583_0001_00_000002
2016-05-17 20:45:57,866 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517957583_0001_1_01_000000_0] to container: [container_1463517957583_0001_00_000002]
2016-05-17 20:45:57,866 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:57,867 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517957583_0001_00_000002 given task: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,867 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957583_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,867 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517957583_0001_1_01_000000_0] started. Is using containerId: [container_1463517957583_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:57,867 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D32E1B14CEFD419D9F85225A98723848, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0, startTime=1463517957867, containerId=container_1463517957583_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517957583_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:57,868 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:57,868 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:57,868 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:57 Starting to run new task attempt: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,869 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,869 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,870 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:57,870 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:57,870 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:57,870 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:57,870 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:57,870 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple, VertexName: D32E1B14CEFD419D9F85225A98723848, VertexParallelism: 1, TaskAttemptID:attempt_1463517957583_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CE088C9F6B3E48728F84F5607D3748C4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A4B7AEA4155F4E6FA46F24EE0D2D08B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:57,870 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:57,871 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,871 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:57,871 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,872 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CE088C9F6B3E48728F84F5607D3748C4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:57,872 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:57,872 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,874 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:57,874 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:57,875 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,875 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,875 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,876 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517957583_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:57,878 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:57,880 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A4B7AEA4155F4E6FA46F24EE0D2D08B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:57,880 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:57,882 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A4B7AEA4155F4E6FA46F24EE0D2D08B9
2016-05-17 20:45:57,882 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:57,882 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A4B7AEA4155F4E6FA46F24EE0D2D08B9
2016-05-17 20:45:57,892 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D32E1B14CEFD419D9F85225A98723848, ordinal: 1
2016-05-17 20:45:57,892 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D32E1B14CEFD419D9F85225A98723848, mem on start (mb), free: 263, total: 655, max: 672
2016-05-17 20:45:57,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:57,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:57,895 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:57,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A4B7AEA4155F4E6FA46F24EE0D2D08B9
2016-05-17 20:45:57,895 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:57,895 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:57,895 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:57,895 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:57,896 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:57,896 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:57,896 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:57,896 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CE088C9F6B3E48728F84F5607D3748C4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:57,896 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A4B7AEA4155F4E6FA46F24EE0D2D08B9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:57,896 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:57,896 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CE088C9F6B3E48728F84F5607D3748C4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:57,896 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,897 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:57,897 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:57,897 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:57,897 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:57,898 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CE088C9F6B3E48728F84F5607D3748C4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:57,898 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:57,898 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:57,899 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:57,899 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:57,900 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:57,901 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,906 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:57,906 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,908 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517957583_0001_1_00_000000_0_10089, runDuration: 50564, ]
2016-05-17 20:45:57,908 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:57,909 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: AF0DE59E97534EF6B52278B068B16535
2016-05-17 20:45:57,909 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517957583_0001_1_00_000000_0_10089] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463517957909, TimeTaken=1, Rate=0.19 MB/s
2016-05-17 20:45:57,909 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"], id: A4B7AEA4155F4E6FA46F24EE0D2D08B9
2016-05-17 20:45:57,909 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,909 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-17 20:45:57,910 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:57,910 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] AF0DE59E97534EF6B52278B068B16535, for 1 inputs
2016-05-17 20:45:57,910 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CE088C9F6B3E48728F84F5607D3748C4] #14 in 2ms
2016-05-17 20:45:57,910 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CE088C9F6B3E48728F84F5607D3748C4] #14, status:false, isInterrupted:false
2016-05-17 20:45:57,912 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:57,913 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 20:45:57,913 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:57,913 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:57,914 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 20:45:57,914 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D32E1B14CEFD419D9F85225A98723848, all 1 inputs ready in: 00:00:00.004
2016-05-17 20:45:57,915 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,916 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@31323d55; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5958b0ae
2016-05-17 20:45:57,916 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:57,919 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:57,919 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:57,920 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,920 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517957583_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:57,921 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,921 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D32E1B14CEFD419D9F85225A98723848, mem on close (mb), free: 261, total: 655, max: 672
2016-05-17 20:45:57,922 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,922 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:57,923 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:57,923 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:57,923 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:57,923 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:57 Completed running task attempt: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,925 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:57,926 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517957914, Process_Duration=7, Process_End_Time=1463517957921, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 20:45:57,926 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D32E1B14CEFD419D9F85225A98723848, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0, startTime=1463517957867, finishTime=1463517957926, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957914, Process_Duration=7, Process_End_Time=1463517957921, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:57,926 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-05-17 20:45:57,926 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:57,926 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,926 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:57,927 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,927 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:57,927 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:57,928 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_FINISHED]: vertexName=D32E1B14CEFD419D9F85225A98723848, taskId=task_1463517957583_0001_1_01_000000, startTime=1463517957867, finishTime=1463517957927, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463517957583_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957914, Process_Duration=7, Process_End_Time=1463517957921, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:57,928 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957583_0001_00_000002
2016-05-17 20:45:57,928 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:57,928 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] : 1
2016-05-17 20:45:57,928 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,929 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:57,929 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_FINISHED]: vertexName=D32E1B14CEFD419D9F85225A98723848, vertexId=vertex_1463517957583_0001_1_01, initRequestedTime=1463517957755, initedTime=1463517957763, startRequestedTime=1463517957777, startedTime=1463517957777, finishTime=1463517957928, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957914, Process_Duration=7, Process_End_Time=1463517957921, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463517957867, firstTasksToStart=[ task_1463517957583_0001_1_01_000000 ], lastTaskFinishTime=1463517957926, lastTasksToFinish=[ task_1463517957583_0001_1_01_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517957583_0001_1_01_000000 ], longestDurationTasks=[ task_1463517957583_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:57,929 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:57,930 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:57,930 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517957583_0001_00_000002, stoppedTime=1463517957930, exitStatus=0
2016-05-17 20:45:57,930 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:57,930 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:57,931 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:57,931 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517957583_0001_1
2016-05-17 20:45:57,931 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517957583_0001_1
2016-05-17 20:45:57,931 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A4B7AEA4155F4E6FA46F24EE0D2D08B9 for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,939 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517957583_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:57,939 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957583_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:57,939 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517957583_0001_00_000002, Exiting
2016-05-17 20:45:57,939 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517957583_0001_00_000002
2016-05-17 20:45:57,939 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517957583_0001_00_000002
2016-05-17 20:45:57,939 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,939 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517957583_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:57,940 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517957583_0001_00_000002 completed successfully
2016-05-17 20:45:57,942 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517957583_0001_1, queueSize=0
2016-05-17 20:45:57,942 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:57,943 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517957583_0001_1, startTime=1463517957752, finishTime=1463517957940, timeTaken=188, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1373634560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035915752, Process_Duration=9, Process_End_Time=2927035915761, Read_Duration=3, Tuples_Read=20, Tuples_Written=18, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 20:45:57,943 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517957583_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:57,943 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957583_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:57,943 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517957583_0001_00_000002
2016-05-17 20:45:57,944 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:57 Completed Dag: dag_1463517957583_0001_1
2016-05-17 20:45:57,945 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517957583_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:57,946 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517957583_0001_1_post
2016-05-17 20:45:57,947 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,947 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:57,951 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463517957583_0001
2016-05-17 20:45:57,953 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:57,953 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:57,953 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:57,953 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:57,953 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:57,954 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:57,955 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:57,955 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:57,955 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:57,955 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:57,956 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:57,956 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:57,956 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:57,961 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001
2016-05-17 20:45:57,961 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple/_temporary
2016-05-17 20:45:57,962 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.398
2016-05-17 20:45:57,993 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimple/simple

Standard error

2016-05-17 20:45:45 Running Dag: dag_1463517944674_0001_1
2016-05-17 20:45:46 Starting to run new task attempt: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46 Completed running task attempt: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46 Starting to run new task attempt: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46 Completed running task attempt: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46 Starting to run new task attempt: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46 Completed running task attempt: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46 Completed Dag: dag_1463517944674_0001_1
2016-05-17 20:45:47 Running Dag: dag_1463517947378_0001_1
2016-05-17 20:45:47 Starting to run new task attempt: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47 Completed running task attempt: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47 Starting to run new task attempt: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47 Completed running task attempt: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47 Completed Dag: dag_1463517947378_0001_1
2016-05-17 20:45:48 Running Dag: dag_1463517948272_0001_1
2016-05-17 20:45:48 Starting to run new task attempt: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48 Completed running task attempt: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48 Completed Dag: dag_1463517948272_0001_1
2016-05-17 20:45:48 Running Dag: dag_1463517948800_0001_1
2016-05-17 20:45:49 Starting to run new task attempt: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49 Completed running task attempt: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49 Completed Dag: dag_1463517948800_0001_1
2016-05-17 20:45:49 Running Dag: dag_1463517949453_0001_1
2016-05-17 20:45:49 Starting to run new task attempt: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49 Starting to run new task attempt: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49 Completed running task attempt: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49 Completed running task attempt: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:50 Starting to run new task attempt: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50 Completed running task attempt: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50 Completed Dag: dag_1463517949453_0001_1
2016-05-17 20:45:50 Running Dag: dag_1463517950564_0001_1
2016-05-17 20:45:50 Starting to run new task attempt: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50 Completed running task attempt: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50 Starting to run new task attempt: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51 Completed running task attempt: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51 Completed Dag: dag_1463517950564_0001_1
2016-05-17 20:45:51 Running Dag: dag_1463517951626_0001_1
2016-05-17 20:45:51 Starting to run new task attempt: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:52 Completed running task attempt: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:52 Starting to run new task attempt: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52 Completed running task attempt: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52 Completed Dag: dag_1463517951626_0001_1
2016-05-17 20:45:52 Running Dag: dag_1463517952642_0001_1
2016-05-17 20:45:52 Starting to run new task attempt: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52 Completed running task attempt: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52 Completed Dag: dag_1463517952642_0001_1
2016-05-17 20:45:53 Running Dag: dag_1463517953104_0001_1
2016-05-17 20:45:53 Starting to run new task attempt: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53 Completed running task attempt: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53 Starting to run new task attempt: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53 Completed running task attempt: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53 Completed Dag: dag_1463517953104_0001_1
2016-05-17 20:45:53 Running Dag: dag_1463517953773_0001_1
2016-05-17 20:45:54 Starting to run new task attempt: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54 Starting to run new task attempt: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54 Completed running task attempt: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54 Completed running task attempt: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54 Starting to run new task attempt: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54 Completed running task attempt: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54 Completed Dag: dag_1463517953773_0001_1
May 17, 2016 8:45:54 PM com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair execute
SEVERE: RuntimeException while executing runnable com.google.common.util.concurrent.Futures$6@3609be45 with executor java.util.concurrent.ThreadPoolExecutor@39aabfba[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
java.util.concurrent.RejectedExecutionException: Task com.google.common.util.concurrent.Futures$6@3609be45 rejected from java.util.concurrent.ThreadPoolExecutor@39aabfba[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
	at com.google.common.util.concurrent.ListenableFutureTask.done(ListenableFutureTask.java:86)
	at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:380)
	at java.util.concurrent.FutureTask.set(FutureTask.java:229)
	at java.util.concurrent.FutureTask.run(FutureTask.java:270)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

2016-05-17 20:45:54 Running Dag: dag_1463517954832_0001_1
2016-05-17 20:45:55 Starting to run new task attempt: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55 Completed running task attempt: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55 Completed Dag: dag_1463517954832_0001_1
2016-05-17 20:45:55 Running Dag: dag_1463517955327_0001_1
2016-05-17 20:45:55 Starting to run new task attempt: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55 Completed running task attempt: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55 Starting to run new task attempt: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55 Completed running task attempt: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55 Completed Dag: dag_1463517955327_0001_1
2016-05-17 20:45:56 Running Dag: dag_1463517955973_0001_1
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56 Completed Dag: dag_1463517955973_0001_1
2016-05-17 20:45:56 Running Dag: dag_1463517956582_0001_1
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56 Completed Dag: dag_1463517956582_0001_1
2016-05-17 20:45:57 Running Dag: dag_1463517957121_0001_1
2016-05-17 20:45:57 Starting to run new task attempt: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57 Completed running task attempt: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57 Completed Dag: dag_1463517957121_0001_1
2016-05-17 20:45:57 Running Dag: dag_1463517957583_0001_1
2016-05-17 20:45:57 Starting to run new task attempt: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57 Completed running task attempt: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57 Starting to run new task attempt: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57 Completed running task attempt: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57 Completed Dag: dag_1463517957583_0001_1